{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1 align=\"center\"> Classification Trees using Python </h1>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.datasets import load_iris\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "from sklearn.model_selection import train_test_split\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from sklearn import tree"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Load the Data\n",
    "The Iris dataset is one of datasets scikit-learn comes with that do not require the downloading of any file from some external website. The code below loads the iris dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sepal length (cm)</th>\n",
       "      <th>sepal width (cm)</th>\n",
       "      <th>petal length (cm)</th>\n",
       "      <th>petal width (cm)</th>\n",
       "      <th>target</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5.1</td>\n",
       "      <td>3.5</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4.9</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4.7</td>\n",
       "      <td>3.2</td>\n",
       "      <td>1.3</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4.6</td>\n",
       "      <td>3.1</td>\n",
       "      <td>1.5</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5.0</td>\n",
       "      <td>3.6</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   sepal length (cm)  sepal width (cm)  petal length (cm)  petal width (cm)  \\\n",
       "0                5.1               3.5                1.4               0.2   \n",
       "1                4.9               3.0                1.4               0.2   \n",
       "2                4.7               3.2                1.3               0.2   \n",
       "3                4.6               3.1                1.5               0.2   \n",
       "4                5.0               3.6                1.4               0.2   \n",
       "\n",
       "   target  \n",
       "0       0  \n",
       "1       0  \n",
       "2       0  \n",
       "3       0  \n",
       "4       0  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = load_iris()\n",
    "df = pd.DataFrame(data.data, columns=data.feature_names)\n",
    "df['target'] = data.target\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Splitting Data into Training and Test Sets\n",
    "One of the benefits of Decision Trees is that you don't have to standardize your data unlike PCA and logistic regression which are [sensitive to effects of not standardizing your data](https://scikit-learn.org/stable/auto_examples/preprocessing/plot_scaling_importance.html#sphx-glr-auto-examples-preprocessing-plot-scaling-importance-py)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# test_size: what proportion of original data is used for test set\n",
    "X_train, X_test, Y_train, Y_test = train_test_split(df[data.feature_names], df['target'], random_state=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Ignore these Cells (just to show train test split for the blog)\n",
    "\n",
    "A relatively new feature of pandas is conditional formatting. https://pandas.pydata.org/pandas-docs/stable/user_guide/style.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train['split'] = 'train'\n",
    "X_test['split'] = 'test'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train['target'] = Y_train\n",
    "X_test['target'] = Y_test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "fullDF = pd.concat([X_train, X_test], axis = 0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sepal length (cm)</th>\n",
       "      <th>sepal width (cm)</th>\n",
       "      <th>petal length (cm)</th>\n",
       "      <th>petal width (cm)</th>\n",
       "      <th>split</th>\n",
       "      <th>target</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>61</th>\n",
       "      <td>5.9</td>\n",
       "      <td>3.0</td>\n",
       "      <td>4.2</td>\n",
       "      <td>1.5</td>\n",
       "      <td>train</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>92</th>\n",
       "      <td>5.8</td>\n",
       "      <td>2.6</td>\n",
       "      <td>4.0</td>\n",
       "      <td>1.2</td>\n",
       "      <td>train</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>112</th>\n",
       "      <td>6.8</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.5</td>\n",
       "      <td>2.1</td>\n",
       "      <td>train</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4.7</td>\n",
       "      <td>3.2</td>\n",
       "      <td>1.3</td>\n",
       "      <td>0.2</td>\n",
       "      <td>train</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>141</th>\n",
       "      <td>6.9</td>\n",
       "      <td>3.1</td>\n",
       "      <td>5.1</td>\n",
       "      <td>2.3</td>\n",
       "      <td>train</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     sepal length (cm)  sepal width (cm)  petal length (cm)  petal width (cm)  \\\n",
       "61                 5.9               3.0                4.2               1.5   \n",
       "92                 5.8               2.6                4.0               1.2   \n",
       "112                6.8               3.0                5.5               2.1   \n",
       "2                  4.7               3.2                1.3               0.2   \n",
       "141                6.9               3.1                5.1               2.3   \n",
       "\n",
       "     split  target  \n",
       "61   train       1  \n",
       "92   train       1  \n",
       "112  train       2  \n",
       "2    train       0  \n",
       "141  train       2  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fullDF.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "fullDFsplit = fullDF.copy()\n",
    "fullDF = fullDF.drop(columns = ['split'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style  type=\"text/css\" >\n",
       "    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row0_col0 {\n",
       "            background-color:  #40E0D0;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row0_col1 {\n",
       "            background-color:  #40E0D0;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row0_col2 {\n",
       "            background-color:  #40E0D0;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row0_col3 {\n",
       "            background-color:  #40E0D0;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row0_col4 {\n",
       "            background-color:  #FFD700;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row1_col0 {\n",
       "            background-color:  #40E0D0;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row1_col1 {\n",
       "            background-color:  #40E0D0;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row1_col2 {\n",
       "            background-color:  #40E0D0;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row1_col3 {\n",
       "            background-color:  #40E0D0;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row1_col4 {\n",
       "            background-color:  #FFD700;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row2_col0 {\n",
       "            background-color:  #40E0D0;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row2_col1 {\n",
       "            background-color:  #40E0D0;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row2_col2 {\n",
       "            background-color:  #40E0D0;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row2_col3 {\n",
       "            background-color:  #40E0D0;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row2_col4 {\n",
       "            background-color:  #FFD700;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row3_col0 {\n",
       "            background-color:  #40E0D0;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row3_col1 {\n",
       "            background-color:  #40E0D0;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row3_col2 {\n",
       "            background-color:  #40E0D0;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row3_col3 {\n",
       "            background-color:  #40E0D0;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row3_col4 {\n",
       "            background-color:  #FFD700;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row4_col0 {\n",
       "            background-color:  #40E0D0;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row4_col1 {\n",
       "            background-color:  #40E0D0;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row4_col2 {\n",
       "            background-color:  #40E0D0;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row4_col3 {\n",
       "            background-color:  #40E0D0;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row4_col4 {\n",
       "            background-color:  #FFD700;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row5_col0 {\n",
       "            background-color:  #40E0D0;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row5_col1 {\n",
       "            background-color:  #40E0D0;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row5_col2 {\n",
       "            background-color:  #40E0D0;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row5_col3 {\n",
       "            background-color:  #40E0D0;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row5_col4 {\n",
       "            background-color:  #FFD700;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row6_col0 {\n",
       "            background-color:  #40E0D0;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row6_col1 {\n",
       "            background-color:  #40E0D0;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row6_col2 {\n",
       "            background-color:  #40E0D0;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row6_col3 {\n",
       "            background-color:  #40E0D0;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row6_col4 {\n",
       "            background-color:  #FFD700;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row7_col0 {\n",
       "            background-color:  #00FFFF;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row7_col1 {\n",
       "            background-color:  #00FFFF;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row7_col2 {\n",
       "            background-color:  #00FFFF;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row7_col3 {\n",
       "            background-color:  #00FFFF;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row7_col4 {\n",
       "            background-color:  #FFFF00;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row8_col0 {\n",
       "            background-color:  #00FFFF;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row8_col1 {\n",
       "            background-color:  #00FFFF;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row8_col2 {\n",
       "            background-color:  #00FFFF;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row8_col3 {\n",
       "            background-color:  #00FFFF;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row8_col4 {\n",
       "            background-color:  #FFFF00;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row9_col0 {\n",
       "            background-color:  #40E0D0;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row9_col1 {\n",
       "            background-color:  #40E0D0;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row9_col2 {\n",
       "            background-color:  #40E0D0;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row9_col3 {\n",
       "            background-color:  #40E0D0;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }    #T_5d8fe69c_b3d7_11e9_93d3_186590dea421row9_col4 {\n",
       "            background-color:  #FFD700;\n",
       "            border-color:  black;\n",
       "            border:  1px solid black;\n",
       "        }</style><table id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421\" ><thead>    <tr>        <th class=\"blank level0\" ></th>        <th class=\"col_heading level0 col0\" >sepal length (cm)</th>        <th class=\"col_heading level0 col1\" >sepal width (cm)</th>        <th class=\"col_heading level0 col2\" >petal length (cm)</th>        <th class=\"col_heading level0 col3\" >petal width (cm)</th>        <th class=\"col_heading level0 col4\" >target</th>    </tr></thead><tbody>\n",
       "                <tr>\n",
       "                        <th id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row0_col0\" class=\"data row0 col0\" >5.1</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row0_col1\" class=\"data row0 col1\" >3.5</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row0_col2\" class=\"data row0 col2\" >1.4</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row0_col3\" class=\"data row0 col3\" >0.2</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row0_col4\" class=\"data row0 col4\" >0</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421level0_row1\" class=\"row_heading level0 row1\" >1</th>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row1_col0\" class=\"data row1 col0\" >4.9</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row1_col1\" class=\"data row1 col1\" >3</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row1_col2\" class=\"data row1 col2\" >1.4</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row1_col3\" class=\"data row1 col3\" >0.2</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row1_col4\" class=\"data row1 col4\" >0</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421level0_row2\" class=\"row_heading level0 row2\" >2</th>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row2_col0\" class=\"data row2 col0\" >4.7</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row2_col1\" class=\"data row2 col1\" >3.2</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row2_col2\" class=\"data row2 col2\" >1.3</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row2_col3\" class=\"data row2 col3\" >0.2</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row2_col4\" class=\"data row2 col4\" >0</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421level0_row3\" class=\"row_heading level0 row3\" >3</th>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row3_col0\" class=\"data row3 col0\" >4.6</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row3_col1\" class=\"data row3 col1\" >3.1</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row3_col2\" class=\"data row3 col2\" >1.5</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row3_col3\" class=\"data row3 col3\" >0.2</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row3_col4\" class=\"data row3 col4\" >0</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421level0_row4\" class=\"row_heading level0 row4\" >4</th>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row4_col0\" class=\"data row4 col0\" >5</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row4_col1\" class=\"data row4 col1\" >3.6</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row4_col2\" class=\"data row4 col2\" >1.4</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row4_col3\" class=\"data row4 col3\" >0.2</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row4_col4\" class=\"data row4 col4\" >0</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421level0_row5\" class=\"row_heading level0 row5\" >5</th>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row5_col0\" class=\"data row5 col0\" >5.4</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row5_col1\" class=\"data row5 col1\" >3.9</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row5_col2\" class=\"data row5 col2\" >1.7</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row5_col3\" class=\"data row5 col3\" >0.4</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row5_col4\" class=\"data row5 col4\" >0</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421level0_row6\" class=\"row_heading level0 row6\" >6</th>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row6_col0\" class=\"data row6 col0\" >4.6</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row6_col1\" class=\"data row6 col1\" >3.4</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row6_col2\" class=\"data row6 col2\" >1.4</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row6_col3\" class=\"data row6 col3\" >0.3</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row6_col4\" class=\"data row6 col4\" >0</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421level0_row7\" class=\"row_heading level0 row7\" >7</th>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row7_col0\" class=\"data row7 col0\" >5</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row7_col1\" class=\"data row7 col1\" >3.4</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row7_col2\" class=\"data row7 col2\" >1.5</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row7_col3\" class=\"data row7 col3\" >0.2</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row7_col4\" class=\"data row7 col4\" >0</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421level0_row8\" class=\"row_heading level0 row8\" >8</th>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row8_col0\" class=\"data row8 col0\" >4.4</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row8_col1\" class=\"data row8 col1\" >2.9</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row8_col2\" class=\"data row8 col2\" >1.4</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row8_col3\" class=\"data row8 col3\" >0.2</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row8_col4\" class=\"data row8 col4\" >0</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421level0_row9\" class=\"row_heading level0 row9\" >9</th>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row9_col0\" class=\"data row9 col0\" >4.9</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row9_col1\" class=\"data row9 col1\" >3.1</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row9_col2\" class=\"data row9 col2\" >1.5</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row9_col3\" class=\"data row9 col3\" >0.1</td>\n",
       "                        <td id=\"T_5d8fe69c_b3d7_11e9_93d3_186590dea421row9_col4\" class=\"data row9 col4\" >0</td>\n",
       "            </tr>\n",
       "    </tbody></table>"
      ],
      "text/plain": [
       "<pandas.io.formats.style.Styler at 0x1a1bf16710>"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\n",
    "def highlight_color(s, fullDFsplit):\n",
    "    '''\n",
    "    highlight the the entire dataframe cyan.\n",
    "    '''\n",
    "\n",
    "    colorDF = s.copy()\n",
    "\n",
    "\n",
    "    colorDF.loc[fullDFsplit['split'] == 'train', data.feature_names] = 'background-color: #40E0D0'\n",
    "\n",
    "\n",
    "    colorDF.loc[fullDFsplit['split'] == 'test', data.feature_names] = 'background-color: #00FFFF'\n",
    "\n",
    "    # #9370DB\n",
    "    # FF D7 00\n",
    "    colorDF.loc[fullDFsplit['split'] == 'train', ['target']] = 'background-color: #FFD700'\n",
    "\n",
    "    # EE82EE\n",
    "    # BD B7 6B\n",
    "    colorDF.loc[fullDFsplit['split'] == 'test', ['target']] = 'background-color: #FFFF00'\n",
    "\n",
    "    return(colorDF)\n",
    "\n",
    "temp = fullDF.sort_index().loc[0:9,:].style.apply(lambda x: highlight_color(x,pd.DataFrame(fullDFsplit['split'])), axis = None)\n",
    "temp.set_properties(**{'border-color': 'black',\n",
    "                       'border': '1px solid black',\n",
    "                       })"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# test_size: what proportion of original data is used for test set\n",
    "X_train, X_test, Y_train, Y_test = train_test_split(df[data.feature_names], df['target'], random_state=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Decision Tree"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Step 1:</b> Import the model you want to use"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In sklearn, all machine learning models are implemented as Python classes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.tree import DecisionTreeClassifier"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Step 2:</b> Make an instance of the Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "clf = DecisionTreeClassifier(max_depth = 2, \n",
    "                             random_state = 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Step 3:</b> Training the model on the data, storing the information learned from the data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Model is learning the relationship between x (features sepal width, sepal height etc) and y (labels-which species of iris)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DecisionTreeClassifier(class_weight=None, criterion='gini', max_depth=2,\n",
       "                       max_features=None, max_leaf_nodes=None,\n",
       "                       min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "                       min_samples_leaf=1, min_samples_split=2,\n",
       "                       min_weight_fraction_leaf=0.0, presort=False,\n",
       "                       random_state=0, splitter='best')"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf.fit(X_train, Y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Step 4:</b> Predict the labels of new data (new flowers)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Uses the information the model learned during the model training process"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([2])"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Returns a NumPy Array\n",
    "# Predict for One Observation (image)\n",
    "clf.predict(X_test.iloc[0].values.reshape(1, -1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Predict for Multiple Observations (images) at Once"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([2, 1, 0, 2, 0, 2, 0, 1, 1, 1])"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf.predict(X_test[0:10])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Measuring Model Performance"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "While there are other ways of measuring model performance (precision, recall, F1 Score, [ROC Curve](https://towardsdatascience.com/receiver-operating-characteristic-curves-demystified-in-python-bd531a4364d0), etc), we are going to keep this simple and use accuracy as our metric. \n",
    "To do this are going to see how the model performs on new data (test set)\n",
    "\n",
    "Accuracy is defined as:\n",
    "(fraction of correct predictions): correct predictions / total number of data points"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.8947368421052632\n"
     ]
    }
   ],
   "source": [
    "score = clf.score(X_test, Y_test)\n",
    "print(score)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Finding the Optimal `max_depth`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "# List of values to try for max_depth:\n",
    "max_depth_range = list(range(1, 6))\n",
    "\n",
    "# List to store the average RMSE for each value of max_depth:\n",
    "accuracy = []\n",
    "\n",
    "for depth in max_depth_range:\n",
    "    \n",
    "    clf = DecisionTreeClassifier(max_depth = depth, \n",
    "                             random_state = 0)\n",
    "    clf.fit(X_train, Y_train)\n",
    "\n",
    "    score = clf.score(X_test, Y_test)\n",
    "    accuracy.append(score)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "ename": "FileNotFoundError",
     "evalue": "[Errno 2] No such file or directory: '../images/max_depth_vs_entropy.png'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mFileNotFoundError\u001b[0m                         Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-19-9a19d163f172>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m     41\u001b[0m \u001b[0max\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mset_ylabel\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Accuracy'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfontsize\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m24\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     42\u001b[0m \u001b[0mfig\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtight_layout\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 43\u001b[0;31m \u001b[0mfig\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msavefig\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'../images/max_depth_vs_entropy.png'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdpi\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m300\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m/anaconda3/envs/decisionTree/lib/python3.7/site-packages/matplotlib/figure.py\u001b[0m in \u001b[0;36msavefig\u001b[0;34m(self, fname, frameon, transparent, **kwargs)\u001b[0m\n\u001b[1;32m   2092\u001b[0m             \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mset_frameon\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mframeon\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2093\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2094\u001b[0;31m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcanvas\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mprint_figure\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfname\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   2095\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2096\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mframeon\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/anaconda3/envs/decisionTree/lib/python3.7/site-packages/matplotlib/backend_bases.py\u001b[0m in \u001b[0;36mprint_figure\u001b[0;34m(self, filename, dpi, facecolor, edgecolor, orientation, format, bbox_inches, **kwargs)\u001b[0m\n\u001b[1;32m   2073\u001b[0m                     \u001b[0morientation\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0morientation\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2074\u001b[0m                     \u001b[0mbbox_inches_restore\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0m_bbox_inches_restore\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2075\u001b[0;31m                     **kwargs)\n\u001b[0m\u001b[1;32m   2076\u001b[0m             \u001b[0;32mfinally\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2077\u001b[0m                 \u001b[0;32mif\u001b[0m \u001b[0mbbox_inches\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0mrestore_bbox\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/anaconda3/envs/decisionTree/lib/python3.7/site-packages/matplotlib/backends/backend_agg.py\u001b[0m in \u001b[0;36mprint_png\u001b[0;34m(self, filename_or_obj, *args, **kwargs)\u001b[0m\n\u001b[1;32m    519\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    520\u001b[0m         \u001b[0;32mwith\u001b[0m \u001b[0mcbook\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_setattr_cm\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrenderer\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdpi\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfigure\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdpi\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;31m \u001b[0m\u001b[0;31m\\\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 521\u001b[0;31m                 \u001b[0mcbook\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mopen_file_cm\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfilename_or_obj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"wb\"\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0mfh\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    522\u001b[0m             _png.write_png(renderer._renderer, fh,\n\u001b[1;32m    523\u001b[0m                             self.figure.dpi, metadata=metadata)\n",
      "\u001b[0;32m/anaconda3/envs/decisionTree/lib/python3.7/contextlib.py\u001b[0m in \u001b[0;36m__enter__\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    110\u001b[0m         \u001b[0;32mdel\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mkwds\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfunc\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    111\u001b[0m         \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 112\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mnext\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgen\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    113\u001b[0m         \u001b[0;32mexcept\u001b[0m \u001b[0mStopIteration\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    114\u001b[0m             \u001b[0;32mraise\u001b[0m \u001b[0mRuntimeError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"generator didn't yield\"\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/anaconda3/envs/decisionTree/lib/python3.7/site-packages/matplotlib/cbook/__init__.py\u001b[0m in \u001b[0;36mopen_file_cm\u001b[0;34m(path_or_file, mode, encoding)\u001b[0m\n\u001b[1;32m    405\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mopen_file_cm\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpath_or_file\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmode\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m\"r\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mencoding\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    406\u001b[0m     \u001b[0;34mr\"\"\"Pass through file objects and context-manage `.PathLike`\\s.\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 407\u001b[0;31m     \u001b[0mfh\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mopened\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mto_filehandle\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpath_or_file\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmode\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mencoding\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    408\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mopened\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    409\u001b[0m         \u001b[0;32mwith\u001b[0m \u001b[0mfh\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/anaconda3/envs/decisionTree/lib/python3.7/site-packages/matplotlib/cbook/__init__.py\u001b[0m in \u001b[0;36mto_filehandle\u001b[0;34m(fname, flag, return_opened, encoding)\u001b[0m\n\u001b[1;32m    390\u001b[0m             \u001b[0mfh\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mbz2\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mBZ2File\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfname\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mflag\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    391\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 392\u001b[0;31m             \u001b[0mfh\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mopen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfname\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mflag\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mencoding\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mencoding\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    393\u001b[0m         \u001b[0mopened\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    394\u001b[0m     \u001b[0;32melif\u001b[0m \u001b[0mhasattr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfname\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'seek'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mFileNotFoundError\u001b[0m: [Errno 2] No such file or directory: '../images/max_depth_vs_entropy.png'"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsgAAAHwCAYAAAC7apkrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzs3Xt4VNX59vHv4nyIgAKKCChaAbFQDEHRn9VitYoWLbVUi6CgaIlIQlBULKB4QEU5BUSlKFDRilixaIEXaqlgFTGAioqCSCCcbEAOJgRIyHr/mAmNMZAMzOy1Z/b9ua5c007W7H2PeYwPizVrGWstIiIiIiISUsV1ABERERERP1GDLCIiIiJSihpkEREREZFS1CCLiIiIiJSiBllEREREpBQ1yCIiIiIipahBFhEREREpxZcNsjFmqDFmtjHmG2OMNcZkH+N1bjbGrDLGFBhjvjXGTDXGND7C2AuMMf80xnxvjNlrjFlgjOlwXG9EREREROKO8eNBIcYYC3wHrAQ6AnuttWdEeI0MYCzwLvAK0AwYDGwEzrfW5pca2xn4N7AFmBR++i7gZOAia+3q43g7IiIiIhJH/Nogn2mt/Sb8vz8DkiJpkI0xjQg1wp8DF1prD4Wf7wbMBf5krR1VavxyoA1wjrV2S/i504A1wDJr7a+i8sZERERExPd8ucSipDk+Dr8B6gATS5rj8HXfAr4BepU8Z4z5CdAJmF3SHIfHbgFmA5cbY5ocZx4RERERiRO+bJCjoFP48YNyvrcMaGOMSarkWENomYeIiIiIBEA11wFipGn4cUs539tCqOltCqytxFiA08q7iTHmDuAOgBo1anRs2bIlRUVFWGupUaMG+fn5NGjQgNzcXJo2bcrGjRs544wz2LBhAy1btiQ7O5vTTz+drVu30rhxY3bv3k3dunU5ePAgxhiqVatGQUEB9erVY+fOnTRp0oTNmzfTokWLw9coeczJyeHUU09lx44d1K9fn3379lG9enUACgsLqVOnDnv27KFRo0Zs27aN5s2b/+gamzZtolmzZmzfvp2GDRuyd+9eateurffko/fUrFkztm3bllDvKRF/TonwnkqulUjvKRF/TvH+nvbv388JJ5yQUO8pEX9OifCetmzZssNaW+5GDeVJ1Aa5TvjxQDnf219mTCRjf8BaOwWYApCcnGxXrlwZeVKRCOTl5ZGUlFTxQJHjpFoTL6jOxCvGmI2RjE/UJRb7wo81y/lerTJjIhl7RJs3b650OJFjNWTIENcRJCBUa+IF1Zn4VaI2yFvDj+UtjTgNsKXGVDQWyl9+8QMtWrSIJJ/IMXn22WddR5CAUK2JF1Rn4leJ2iB/FH68sJzvXQB8Za3Nq8TYzoSa6RUV3XDDhg2RZhSJWK9evSoeJBIFqjXxgupM/MqX+yCXVtE+yMaYFoTWCK+31haGn2tMaB/k1YQO+ii7D/Jwa+2jpa7xEdAaaGOt3Rp+rinwJbDcWnt5RTlTUlJsVlbWMb9PEREREYkNY8wKa21KZcf7cgbZGNPbGDPMGDMMaAzUL/n/xpjeZYb/hdCBHoeXSFhrc4HhwPnAP40xdxhjRgJ/JdT0ji9zjXRCa5CXGmMGGWMGAUsJ/fO5uzKZNYMsXtBsi3hFtSZeUJ2JX/lyBtkY82/g0iN8+11r7S/KGdvSWptd5jp9gAxCs8N7gbeB+621/y3nnhcCjxJagmGB94Gh1tpKbU2hGWQRERERf0qIGWRr7S+steYIX784wtjscq4z3Vr7M2ttLWvtydbaW8trjsNjP7DW/tJam2StPcFae2Vlm2OAnJycSN+mSMQyMjJcR5CAUK2JF1Rn4le+nEGORx06dLAff/yx6xiS4HJzc2ncuNL7nIscM9WaeEF1Jl5JiBnkeLRjxw7XESQApk2b5jqCBIRqTbygOhO/UoMcJfXr13cdQQKga9euriNIQKjWxAuqM/ErNchRsm9fhYftiRw3HWcuXlGtiRdUZ+JXapCjpHr16q4jSACceuqpriNIQKjWxAuqM/ErNcgiIiIiIqWoQY6SwsJC1xEkALZt2+Y6ggRAUVER69ev59ChQ66jSILT7zTxKzXIUVKnTh3XESQAkpOTXUeQBHXgwAFmzpzJRe3aUatGDe5NT6dm9epc1K4dM2fO5MCBA64jSgLS7zTxKzXIUbJnzx7XESQA5s+f7zqCJKDly5dzVtOmvJSayr2ffcZ+a/m+sJD91nLvZ5/xl9RUzmralI8++sh1VEkw+p0mfqWDQqJEB4WIF7SpvkTbRx99xNVduvBCfj7XHmXcXOC2unWZt3gxnTp18iqeJDj9ThOv6KAQR7SOSrwwatQo1xEkgRw4cIDuV11VYXMMcC3wQn4+3a+6SsstJGr0O038Sg1ylDRv3tx1BAmAcePGuY4gCWT27Nm0PXiwwua4xLXAOQcP8vrrr8cylgSIfqeJX1VzHSBRbNiwwXUECYBevXoxc+ZM1zEkQUx+8knuzcuL6DV35uVx/5Ah7Nq1K0apJEhmzJjBLbfc4jqGyI9oDXKUpKSk2KysLNcxREQq5dChQ9SsXp391kY0U1IE1AD0Xw4RiTMRrUHWDHKUaAZZvKAZZImWvLw86lSvTrWDByN6XTWgVpUq3HTrrdSsWTM24SQwFi1axBVXXOE6hgTAM888E9F4zSBHiWaQRSReFBUV8cYbb/CHG27gAJHNlBQBtYzhQGEhVatWjVFCEZHo0i4WjmzatMl1BAmA1NRU1xEkju3cuZMnn3ySM888kxtuuIG6wNsRXuMt4Pxzz1VzLFGh32niV5pBjpLk5GS7cuVK1zEkweXl5ZGUlOQ6hsSZ1atXk5mZycyZM9m/fz8ArVq1onPnzmx94w0WRfBBvSuSkujz3HPcdNNNsYorAaLfaeIVzSA7sn37dtcRJADGjBnjOoLEiUOHDvHmm29y2WWX0b59e6ZOncr+/fvp2rUr8+fPZ82aNUyZMoU1NWowt5LXnAusqVGD3/3ud7GMLgGi32niV/qQXpQ0bNjQdQQJgJ49e7qOID63a9cuXnzxRSZNmkR2djYASUlJ9OnTh7vuuovWrVsfHluzZk3mLFgQ2Ul6Cxbow3kSNfqdJn6lGeQo2bt3r+sIEgCLFy92HUF8as2aNaSmptKsWTPuuecesrOzOfPMMxk3bhybN29m4sSJP2iOS3Tq1Il5ixdz50kn8aukJOYQ+iAe4cc5hJZV3HnSSTpmWqJOv9PErzSDHCW1a9d2HUECoE2bNq4jiI8UFxczb948MjMzWbRo0eHnr7jiCtLS0ujatWulPkzXqVMn1m/dyuuvv85TTzxBj88/p1bVquw/dIjzzz2XAfffz+9+9zvNHEvU6Xea+JUa5CgpKiqqeJDIcdq9e7frCOIDe/bsYfr06UycOJH169cDUKdOHW6++WYGDhxI27ZtI75mzZo1uemmm7jppps4dOgQr732Gr///e+1W4XElH6niV+pQY4S7QYiXigoKHAdQRxau3YtEydOZPr06eSFd544/fTTueuuu7jttts48cQTo3KfqlWrUqVKFTXHEnP6nSZ+pQY5SmrUqOE6ggRAy5YtXUcQjxUXF7Nw4UIyMzOZP3/+4ee7dOlCWloa3bp1i0kjq1oTL6jOxK/0Ib0oyc/Pdx1BAmDZsmWuI4hHvv/+e5555hnatm17eGu2WrVq0a9fPz799FP+9a9/8Zvf/CZms7yqNfGC6kz8SjPIUdKgQQPXESQAunfv7jqCxNj69euZNGkSL7744uHdcZo3b86AAQPo16+fZ1tKqtbEC6oz8SvNIEdJbm6u6wgSAJMnT3YdQWLAWss///lPrr32Ws4++2zGjx/P3r17+fnPf87s2bP55ptvuO+++zzdb121Jl5QnYlf6ajpKOnYsaNdsWKF6xiS4IqKiqhWTX/xkyjy8/OZOXMmmZmZfPHFF0Do8ww9e/YkLS2N8847z1k21Zp4QXUmXtFR045s3LjRdQQJgH79+rmOIFGQnZ3NkCFDaNasGf379+eLL76gadOmPProo+Tk5DBt2jSnzTGo1sQbqjPxK80gR0lKSorNyspyHUNEfMpay7vvvktmZiZ///vfKS4uBuDCCy8kLS2N66+/nurVqztOKSKSmDSD7MiGDRtcR5AA6N27t+sIEqGCggJeeOEFOnToQJcuXZgzZw5Vq1alV69eLF++nPfff58bb7zRd82xak28oDoTv9IMcpRoBllESsvJyWHy5MlMmTKF7777DoBTTjmF1NRU/vjHP9KkSRPHCUVEgkMzyI5kZ2e7jiAB0KdPH9cR5Cistbz33nv8/ve/p2XLljzxxBN89913pKSk8NJLL7Fx40YefPDBuGiOVWviBdWZ+JVmkKNEu1iIF/SJb3/av38/s2bNYsKECaxatQqAatWq0aNHD9LS0rjgggswxjhOGRnVmnhBdSZe0QyyI1u3bnUdQQJg+PDhriNIKVu3bmX48OG0aNGCPn36sGrVKho3bsywYcPIzs7mlVdeoXPnznHXHINqTbyhOhO/0gxylLRv395++umnrmNIgsvJyaF58+auYwSatZYPP/yQCRMm8Prrr1NUVATAeeedR1paGjfeeCO1atVynPL4qdbEC6oz8YpmkB3ZvXu36wgSAHPmzHEdIbAOHjzIzJkzueCCC7jwwgt59dVXsdbSo0cPli5dyooVK+jTp09CNMegWhNvqM7Er7TwJ0rq1q3rOoIEQOfOnV1HCJzt27fz/PPP89xzz7F9+3YATjrpJO644w5SU1Np0aKF44SxoVoTL6jOxK80gxwlBw8edB1BAkD7bXsnKyuL3r1706JFCx566CG2b99Ou3btmDp1Kps3b+bxxx9P2OYYVGviDdWZ+JVmkKMkHj+EI/Gndu3ariMktMLCQv72t7+RmZnJBx98AECVKlXo3r07aWlpXHrppYH5d121Jl5QnYlfqUGOEm1TI15o0KCB6wgJKTc3lylTpjB58uTDO9I0aNCAfv36MWDAAM444wy3AR1QrYkXVGfiV1piESUFBQWuI0gAfPnll64jJJRVq1bRt29fmjdvzrBhw9i6dStt27blueeeY/PmzTz11FOBbI5BtSbeUJ2JX2naM0rq1avnOoIEQJcuXVxHiHtFRUW8+eabZGZmsnTpUiC0RKpbt26kpaXxy1/+MjDLKI5GtSZeUJ2JX2kGOUp27tzpOoIEwCuvvOI6QtzauXMnTz75JGeeeebhrdnq1avHoEGDWLduHXPnzuXyyy9XcxymWhMvqM7Er3RQSJQkJyfblStXuo4hCS4vL4+kpCTXMeLK6tWryczMZObMmezfvx+AVq1akZaWxs0338wJJ5zgOKE/qdbEC6oz8YoOCnFk8+bNriNIAAwZMsR1hLhw6NAh3nzzTS677DLat2/P1KlT2b9/P127dmX+/PmsWbOGAQMGqDk+CtWaeEF1Jn6lGeQoSUlJsVlZWa5jiATarl27ePHFF5k0aRLZ2dkAJCUl0adPHwYOHEirVq3cBhQRESc0g+yINjsXL/Tq1ct1BF/64osvSE1NpVmzZtxzzz1kZ2dz1llnMX78eDZv3szEiRPVHEdItSZeUJ2JX2kGOUo0gyzireLiYubNm0dmZiaLFi06/PwVV1xBWloaV199NVWqaA5AREQ0g+yMZpDFC5ptgT179jBhwgRatWpFt27dWLRoEXXq1KF///58/vnnLFy4kF//+tdqjo+Tak28oDoTv9IMcpRoBlkktr766ismTZrE9OnTycvLA+CMM87grrvu4tZbb+XEE090nFBERPxKM8iO5OTkuI4gAZCRkeE6gqeKi4tZsGABXbt2pU2bNkyaNIm8vDy6dOnCm2++yddff83dd9+t5jgGglZr4obqTPxKM8hR0qFDB/vxxx+7jiEJLjc3l8aNG7uOEXPff/89M2bMYOLEiaxduxaAWrVq0bt3bwYOHEi7du0cJ0x8Qak1cUt1Jl7RDLIjO3bscB1BAmDatGmuI8TU+vXrycjIoFmzZgwcOJC1a9fSvHlznnjiCTZv3syUKVPUHHsk0WtN/EF1Jn5VzXWARFG/fn3XESQAunbt6jpC1Flreeedd5gwYQL/+Mc/KPlbrZ///Oekp6dz3XXXUa2aflV5LRFrTfxHdSZ+pRnkKNm3b5/rCBIAiXSceX5+Ps899xw//elPueKKK3j77bepUaMGffv2ZeXKlSxZsoTrr79ezbEjiVRr4l+qM/Er/ZcnSqpXr+46ggTAqaee6jrCccvOzuaZZ55h6tSp7N69G4CmTZty5513cscdd2g9ok8kQq2J/6nOxK/UIItIzFlreffdd5kwYQJz586luLgYgIsuuoi0tDR++9vf6g+ZIiLiG2qQo6SwsNB1BAmAbdu2uY4QkYKCAl5++WUyMzNZvXo1EPrblptuuomBAwfSqVMnxwnlSOKt1iQ+qc7Er9QgR0mdOnVcR5AASE5Odh2hUnJycpg8eTJTpkzhu+++A6BJkyakpqZyxx130KRJE8cJpSLxUmsS31Rn4lf6kF6U7Nmzx3UECYD58+e7jnBE1lree+89evToQcuWLXniiSf47rvv6NSpEzNnzmTjxo2MGDFCzXGc8HOtSeJQnYlf6aCQKNFBIeIFP26qv3//fl599VUyMzNZtWoVANWqVaNHjx6kpaXRuXNnxwnlWPix1iTxqM7EKzooxBGtoxIvjBo1ynWEw7Zu3crw4cNp0aIFffv2ZdWqVTRu3Jjhw4ezceNGXnnlFTXHccxPtSaJS3UmfqUZ5ChJSUmxWVlZrmOIxJS1lmXLlpGZmcnrr79OUVERAOeddx7p6enccMMN1KpVy3FKERGRH9IMsiMbNmxwHUECoFevXk7ue+DAAWbOnMkFF1zARRddxKuvvoq1lh49erB06VJWrFjBLbfcouY4gbiqNQkW1Zn4lWaQo0QzyJKItm/fzvPPP8+zzz7Lt99+C0DDhg254447SE1NpXnz5o4TioiIVEwzyI5oBlm84NVsy0cffUTv3r1p0aIFDz30EN9++y3t27dn6tSp5OTkMGrUKDXHCU4ze+IF1Zn4lWaQo0QzyBLvCgsL+dvf/kZmZiYffPABAFWqVOG6664jPT2dSy65BGOM45QiIiKR0wyyI5s2bXIdQQIgNTU16tfMzc3lscce44wzzuAPf/gDH3zwAQ0aNGDIkCGsX7+eN954g0svvVTNccDEotZEylKdiV/5cgbZGFMFSAf+CJwB5AKvASOstfmVeP0pwEjgGuAUYDswB3jQWru7nPGtgSeBS4EawMrw2H9VNnNycrJduXJlZYeLHJO8vDySkpKicq1Vq1aRmZnJX//6Vw4cOABA27ZtSUtLo1evXtStWzcq95H4FM1aEzkS1Zl4JVFmkMcBY4EvgIHAbCANeCvcPB+RMeZk4EPgVuDN8Ov/DqQCi40xdcqMPwt4H7gQGA0MAZKA/2eMubyygbdv317ZoSLHbMyYMcf1+qKiIl5//XV+/vOfk5yczPTp0zl48CDdunXjn//8J5999hl//OMf1RzLcdeaSGWozsSvqrkOUJYx5lxCTe0b1trrSz2/AcgEbgReOcolHgBOB3paa/9a6vXvh183GHi01PjHgQZAR2vtx+GxfwE+B54xxrSxlZhmb9iwYeXeoMhx6Nmz5zG9bufOnfz5z39m8uTJ5OTkAFCvXj1uu+02BgwYwFlnnRXNmJIAjrXWRCKhOhO/8uMM8h8AA4wv8/yfgX1ARR957QIUAK+WeX4WsB/oW/KEMaYucC3w75LmGMBamwdMBVoBnSoTeu/evZUZJnJcFi9eHNH4Tz/9lNtvv51mzZoxdOhQcnJyaN26NZMmTWLLli2MHTtWzbGUK9JaEzkWqjPxK9/NIBNqSIuB5aWftNbuN8Z8TMUNa01gf9lZX2ttsTGmADjTGNPIWrsDaB8e/0E511lWKs/ycr7/A7Vr165oiMhxa9OmTYVjDh06xFtvvcWECRP497//ffj5q6++mrS0NK644gqqVPHjn43FTypTayLHS3UmfuXHBrkpsMNae6Cc720BLjLG1LDWHjzC6z8HWhtjOpSeFTbGdABODP/fFsCO8L1KrlvevQBOq0zokiN3RWJp9+4ffcb0sF27dvHCCy/wzDPPkJ2dDUBSUhJ9+/blrrvuolWrVh6llERwtFoTiRbVmfiVH6eR6gDlNccQWiJRMuZIxhOagX7NGHO1MaaFMaYroSUWhWVeX/JY3v0qvJcx5g5jTJYxJuvbb79lyZIlzJ07l1mzZrF8+XIyMzPJyclh6NChFBUV0adPHwB69+4NQJ8+fSgqKjr8V9+ZmZksX76cWbNmMXfuXJYsWcKUKVNYt24dI0eOJC8v7/CWOCWbq5c8ZmRkkJuby+jRo1m9ejUzZsxg4cKFLFy4kBkzZrB69WpGjx5Nbm4uGRkZ5V4jNTWVvLw8Ro4cybp165gyZYrek8/eU0FBwY/e00MPPcR1111HkyZNGDJkCNnZ2TRq1IiHHnqI22+/nbFjxzJq1CjfvqdE/DklwnsqKChIuPeUiD+neH9PCxYsSLj3lIg/p0R4T5Hy3TZvxpjVwMnW2lPK+d5rQA+g5lFmkDHG9CD0gb4m4acOEVpTfDLQHfiZtfZTY8z1wOvAndbaZ8tcoy2h2ejHrbUPVJT7nHPOsWvWrKnMWxQ5ZsuXL+f888+nuLiYefPmkZmZyaJFiw5//1e/+hVpaWl07dpVyyjkuJTUmkgsqc7EK5Fu8+bHJRZbgbbGmJrlLLM4jdDyiyM2xwDW2tnGmDeAdsAJwFfW2v8aY5YDRcDXpe5Vct2ySp4rb/nFj+TnV7g9s8hxW7x4Me+//z6TJk1i/fr1ANSpU4dbbrmFgQMHcs455zhOKIli2bJlalwk5lRn4ld+bJA/An4FnA8sLXnSGFML6AAsqcxFrLWHgNJrkJsA5wHvWmv3hZ9eTWh5xYXlXKJz+LFS50c3aNCgMsNEjtlbb73Fww8/zL59ofI944wzGDhwILfeeqvqT6Kue/furiNIAKjOxK/8+HewswALDCrz/O2E1gO/XPKEMeYsY0yFH4ENHy6SCVQFHit5Pryd21vAL4wxPys1PgnoB6yjEjtYQOi4XpFY2bt3L7fffjv79u3jF7/4BW+++SZff/01gwcPVnMsMTF58mTXESQAVGfiV75bgwxgjJkI3EXoeOh5wDmETtL7D3CZtbY4PC4bON1aa0q9NolQUzsH2ADUJ7S3ckfgT9baUWXu9ZPw+EJCJ/jtJdSMtwOusdb+v8pk7tixo12xYsUxvmORo7v33nt56qmn6Ny5M++//z7GmIpfJHIcioqKqFbNj3/JKIlEdSZeSZSjpgcB9wDnAs8QOj1vIvDrkub4KA4CnwI9gUnAn4CdwFVlm2MAa+3XwP8R2vf4fuBpID88vlLNMcDGjRsrO1QkImvXrmX8+NC5OQ0bNlRzLJ7o16+f6wgSAKoz8StfziDHo5SUFJuVVanlyiIR6datG2+//TZ9+/blxRdfdB1HREQk7iTKDHLc2bBhg+sIkoAWLFjA22+/zQknnMCoUaMO7xMpEmuqNfGC6kz8SjPIUaIZZIm2wsJC2rVrx1dffcXo0aMZMmSI60giIiJxSTPIjpQc7SsSLZMmTeKrr77i7LPPJj09HeDwiUMisaZaEy+ozsSvNIMcJdrFQqLpv//9L2effTZ79+7l7bff5pprrgH0iW/xjmpNvKA6E69oBtmRrVu3VjxIpJKGDRvG3r17ueqqq7j66qsPPz98+HCHqSRIVGviBdWZ+JVmkKOkffv29tNPP3UdQxLAypUrSUlJoWrVqqxevZo2bf53Fk5OTg7Nmzd3mE6CQrUmXlCdiVc0g+zI7t27XUeQBGCtJT09HWstAwcO/EFzDDBnzhxHySRoVGviBdWZ+JUa5CipW7eu6wiSAGbNmsV7771H48aNGTFixI++37lzZwepJIhUa+IF1Zn4lRrkKDl48KDrCBLn8vPzD2/l9thjj9GgQYMfjdF+2+IV1Zp4QXUmfqUGOUp0/K8cr9GjR7N582bOO+88br311nLH1K5d2+NUElSqNfGC6kz8Sg1ylGibGjkeGzduZPTo0QBkZmZStWrVcseVN6ssEguqNfGC6kz8Sg1ylBQUFLiOIHFsyJAh7N+/nxtvvJGLL774iOO+/PJLD1NJkKnWxAuqM/ErNchRUq9ePdcRJE79+9//Zvbs2dSuXfvwLPKRdOnSxaNUEnSqNfGC6kz8Sg1ylOzcudN1BIlDhw4dOnyM9P3331/hfqCvvPKKF7FEVGviCdWZ+JUOComS5ORku3LlStcxJM4899xzpKamcvrpp7NmzZoKP7CSl5dHUlKSR+kkyFRr4gXVmXhFB4U4snnzZtcRJM7s2rWLYcOGAfD0009X6tPcJdvAicSaak28oDoTv9IMcpSkpKTYrKws1zEkjqSnp5OZmcmll17K4sWLtVWgiIhIjGgG2RFtdi6R+Pzzz3nmmWeoUqUKEyZMqHRz3KtXrxgnEwlRrYkXVGfiV5pBjhLNIEtlWWu58sorWbRoEf379+fZZ591HUlERCShaQbZEc0gS2XNnTuXRYsW0aBBAx555JGIXqvZFvGKak28oDoTv9IMcpRoBlkqY//+/Zx77rl88803ZGZmMnDgQNeRREREEp5mkB3JyclxHUHiwPjx4/nmm29o27Yt/fv3j/j1GRkZMUgl8mOqNfGC6kz8SjPIUdKhQwf78ccfu44hPrZ161ZatWpFfn4+ixYt4vLLL4/4Grm5uTRu3DgG6UR+SLUmXlCdiVc0g+zIjh07XEcQnxs6dCj5+flcd911x9QcA0ybNi3KqUTKp1oTL6jOxK/UIEdJ/fr1XUcQH/vwww/5y1/+Qo0aNRgzZswxX6dr165RTCVyZKo18YLqTPxKDXKU7Nu3z3UE8ani4mLS0tIAGDx4MGedddYxX0vHmYtXVGviBdWZ+JUa5CipXr266wjiUy+99BLLly/n1FNP5YEHHjiua5166qlRSiVydKo18YLqTPxKDbJIDH3//ffcf//9ADz55JOccMIJjhOJiIhIRdQgR0lhYaHrCOJDjz32GNu3b+eCCy7gpptuOu483JD8AAAgAElEQVTrbdu2LQqpRCqmWhMvqM7Er9QgR0mdOnVcRxCf+frrrxk3bhwAmZmZVKly/P+6JScnH/c1RCpDtSZeUJ2JX6lBjpI9e/a4jiA+c/fdd3Pw4EFuueUWzj///Khcc/78+VG5jkhFVGviBdWZ+JUOCokSHRQipS1cuJArr7ySpKQk1q5dG7UPomhTffGKak28oDoTr+igEEe0jkpKFBYWMmjQIACGDx8e1U9pjxo1KmrXEjka1Zp4QXUmfqUZ5ChJSUmxWVlZrmOID0yYMIFBgwbxk5/8hM8++4yaNWu6jiQiIhJomkF2ZMOGDa4jiA/k5uby4IMPAjB27NioN8e9evWK6vVEjkS1Jl5QnYlfaQY5SjSDLAD9+/fn+eef51e/+hULFizAGOM6koiISOBpBtkRzSDLxx9/zJQpU6hatSrjx4+PSXOs2RbximpNvKA6E7/SDHKUaAY52Ky1XHrppSxdupT09HTGjx/vOpKIiIiEaQbZkU2bNrmOIA7Nnj2bpUuX0qhRIx566KGY3Sc1NTVm1xYpTbUmXlCdiV9pBjlKkpOT7cqVK13HEAf27dtHmzZtyMnJ4fnnn+eOO+6I2b3y8vJISkqK2fVFSqjWxAuqM/GKZpAd2b59u+sI4shTTz1FTk4OHTp04LbbbovpvcaMGRPT64uUUK2JF1Rn4ldqkKOkYcOGriOIA5s2beLJJ58EQvsfV61aNab369mzZ0yvL1JCtSZeUJ2JX6lBjpK9e/e6jiAO3HvvvRQUFPD73/+eSy65JOb3W7x4cczvIQKqNfGG6kz8Sg1ylNSuXdt1BPHYkiVLmDVrFrVr1+app57y5J5t2rTx5D4iqjXxgupM/EoNcpQUFRW5jiAeOnToEGlpaQDcd999tGjRwpP77t6925P7iKjWxAuqM/ErNchRot1AguWFF17gk08+oXnz5gwZMsSz+xYUFHh2Lwk21Zp4QXUmfqUGOUpq1KjhOoJ4ZNeuXfzpT38C4Omnn6ZOnTqe3btly5ae3UuCTbUmXlCdiV+pQY6S/Px81xHEIw8//DA7duzgkksuoUePHp7ee9myZZ7eT4JLtSZeUJ2JX+mgkChp3769/fTTT13HkBhbs2YN7du3p7i4mBUrVtChQwdP75+Tk0Pz5s09vacEk2pNvKA6E6/ooBBHcnNzXUeQGLPWMmjQIIqKirj99ts9b44BJk+e7Pk9JZhUa+IF1Zn4lWaQo6Rjx452xYoVrmNIDL311ltce+211K9fn3Xr1tG4cWPPMxQVFVGtWjXP7yvBo1oTL6jOxCuaQXZk48aNriNIDB04cICMjAwARo4c6aQ5BujXr5+T+0rwqNbEC6oz8SvNIEdJSkqKzcrKch1DYmT06NHcd999nHPOOXzyySdUr17ddSQRERGpJM0gO7JhwwbXESRGtm3bxiOPPALAuHHjnDbHvXv3dnZvCRbVmnhBdSZ+pRnkKNEMcuLq27cv06dPp1u3bsydO9d1HBEREYmQZpAdyc7Odh1BYmD58uVMnz6dGjVqMHbsWNdx6NOnj+sIEhCqNfGC6kz8SjPIUaJdLBJPcXExF110ER9++CH33XcfTzzxhOtI+sS3eEa1Jl5QnYlXNIPsyNatW11HkCh7+eWX+fDDD2nSpMnho6VdGz58uOsIEhCqNfGC6kz8SjPIUaKT9BLL999/T+vWrdm2bRvTp0/nlltucR0J0KlT4h3VmnhBdSZe0QyyI7t373YdQaLo8ccfZ9u2bZx//vm++pT1nDlzXEeQgFCtiRdUZ+JXapCjpG7duq4jSJSsX7+eMWPGAJCZmUmVKv7516Rz586uI0hAqNbEC6oz8Sv//Jc/zh08eNB1BImSu+++m4MHD3LzzTdzwQUXuI7zA9pvW7yiWhMvqM7Er9QgR4kxxnUEiYJFixbx97//nbp16/L444+7jvMjtWvXdh1BAkK1Jl5QnYlfqUGOEm1TE/8KCwsZNGgQAMOGDaNp06aOE/1YgwYNXEeQgFCtiRdUZ+JXapCjpKCgwHUEOU7PPfccX3zxBWeeeebhRtlvvvzyS9cRJCBUa+IF1Zn4lRrkKKlXr57rCHIcduzYwYgRIwAYO3YstWrVcpyofF26dHEdQQJCtSZeUJ2JX0XUIBtjzo5VkHi3c+dO1xHkOIwYMYLdu3dz+eWXc+2117qOc0SvvPKK6wgSEKo18YLqTPwqooNCjDGHgPeAF4HZ1tp9sQoWb5KTk+3KlStdx5Bj8Mknn5CcnIwxhk8//ZS2bdu6jnREeXl5JCUluY4hAaBaEy+ozsQrsT4oxAA/J9QgbzPG/NkYc2GE10hImzdvdh1BjoG1lvT0dIqLixkwYICvm2OAIUOGuI4gAaFaEy+ozsSvIp1BbgncCtwMNAdKXvwVoab5JWvtt9EOGQ9SUlJsVlaW6xgSoddff50ePXrQsGFD1q1bx4knnug6koiIiERZTGeQrbUbrLXDgTOAK4HXgANAG+BJIMcY86Yxppsxpmok14532uw8/hQUFHDPPfcA8Oijj8ZFc9yrVy/XESQgVGviBdWZ+FVEM8jlXsCY+kAvoC+QHH7aAv8F/gJMs9ZGtI+LMaYKkA78kVAznkuoGR9hrc2vxOuTgDTgD+HXHwDWAlOAGbbMmzbGXAA8BlwQzv4+cL+19uPKZtYMcvx55JFHGDFiBO3bt2flypVUrRqoP9OJiIgERqzXIP+ItXaPtfaZ8E3bA+OBHcApwD3A58aY/xhjbjHG1KzkZccBY4EvgIHAbEIN71vh5vmIwt+fDzwCfATcDTwKVAWmAU+UGd8ZeBdoCYwAHgTOBpYaY9pVMq9mkONMTk7O4ZPyMjMz46Y51myLeEW1Jl5QnYlfHfcM8g8uZswpwC2E1imfTehDfSUs8C0w1Fo74yjXOBdYDcyx1l5f6vmBQCZwk7X2iPvChD80+D4w3lqbUer5GsCXwEnW2galnl9OaInIOdbaLeHnTgPWAMustb+qzHvXDHJ8+cMf/sCrr75Kjx49eO2111zHERERkRjyfAbZGFPNGNPdGDMXyAEeB1oRWhbxNHARMBzYCDQBXjTG3HqUS/6BUGM9vszzfwb2EVrOcTQlJ3ZsLf2ktfYgoZntw0s0jDE/AToR2rJuS6mxWwjNWl9ujGlSwf2A0IykxIelS5fy6quvUqtWLZ566inXcSKSkZFR8SCRKFCtiRdUZ+JXx9wgG2PaGWPGEWpEXwd+Hb7eQuB3QDNr7b3W2mXW2seAnxBa9mAILXs4kk5AMbC89JPW2v3Ax+HvH81yYDdwrzGmhzGmhTGmtTHmcaAj8FCZewF8UM51loWzdqzgfgCceuqplRkmjh06dIj09HQA7r33Xk4//XTHiSLzwAMPuI4gAaFaEy+ozsSvIj1J70RjzABjTBahZjUNaARsBkYCLa21Xa21b1hri0q/1lpbbK19ENgLnHWU2zQFdlhrD5TzvS1Ao/ByiXJZa3cB1wLfEfpg30ZCSysGANdba/9c5l4l1y3vXgCnHelexpg7jDFZxpistWvXsmTJEubOncusWbNYvnw5mZmZ5OTkMHToUIqKiujTpw8AvXv3BqBPnz4UFRUxdOhQcnJyyMzMZPny5cyaNYu5c+eyZMkSpkyZwrp16xg5ciR5eXmkpqYC/1u3VfKYkZFBbm4uo0ePZvXq1cyYMYOFCxeycOFCZsyYwerVqxk9ejS5ubmH/8Re9hqpqank5eUxcuRI1q1bx5QpUxLuPfXt25dVq1bRsGFDzjzzzLh7T9OmTQvEz0nvyf17mjZtWsK9p0T8OcX7e7rrrrsS7j0l4s8pEd5TpCLdB7kAqEFoZrUQeAuYCvy/sjtDHOUa2UBza225n4oyxqwHqltrW5Tzvb8AvYETrbW7j3KP84BhwDeE1iOfRKhBbgNcZ61dFB43HHgY+KW19l9lrnEZ8A6QYa0tu9zjR84991z7+eefVzRMHNq9ezetWrUiNzeXv/71r9x4442uI0Vs9erVtGtX6c+Oihwz1Zp4QXUmXon1GuSawDrgPkJLKH5nrV1Q2eY47AbgsqN8f1/4PuWpVWpMucI7T7wPLLLWDrHWzrHWvgBcDGwH/lxqj+aS65R3vwrv9YPQ+3Tqtt898sgj5ObmcvHFF3PDDTe4jnNMdJy5eEW1Jl5QnYlfVYtw/KXW2qXHc0Nr7YcVDNkKtDXG1CxnmcVphJZfHDzK6zMINbezy9x3nzHmH8BdhPZGXs//PshX3jKKkufKW37xI9WrV6/MMHHkyy+/JDMzE2PM4cd4pLXu4hXVmnhBdSZ+FelJesfVHFfSR4RynV/6SWNMLaADUNFeaiWNbXlLOKqVefwo/HhhOWM7E9qabkUF9xOfs9aSkZFBUVER/fr147zzznMdSURERHzsWD6kd7Mxplslxl4bHtugorFlzCLUmA4q8/ztQB3g5VL3OMsY06bMuC/Cj33K5GkAXAfsIjR7jLX2a0INdw9jTNNSY5sCPYB/WWu3VyZ0YWFhZYaJA/PmzWPBggXUr1+fRx991HWc47Jt2zbXESQgVGviBdWZ+FWka5BvIXQaXXJFA4FLwmMjOibHWrsaeAb4rTHmDWNMP2PMGEIn670LlD4k5B1CB3qUNp7QDhZPGGNeMsb0N8Y8AKwCTgWGldlhI53QGuSlxphBxphBwFJC/2yOth3dD9SpUyeStykeOXjw4OFP2z744IOcfPLJjhMdn+TkyvyrJ3L8VGviBdWZ+FWkDXL38OPso44KmUFot4vfRngPCM0e3wOcS6hZvhGYCPzaWlt8tBdaazcSWp7xEtAl/Lr7CR1icr21dnKZ8e8DvwCyCR1J/QjwNXCJtfaTygbes2dPZYeKhzIzM1m3bh2tW7dmwIABruMct/nz57uOIAGhWhMvqM7EryLd5m0zcApQu+w+x+WMrQ4UAFvL27It0XTo0MF+/PHHrmNIKdu3b6dVq1Z8//33zJ8/n6uuusp1pOOWm5tL48aNXceQAFCtiRdUZ+KVWG/z1gjYW1FzDGCtLQT2APH9d9qVpHVU/vPAAw/w/fff8+tf/zohmmOAUaNGuY4gAaFaEy+ozsSvIp1B3gY0Bk6y1u6tYGx9YCew01p7ynGljAMpKSk2K6uiDTbEKx999BHnn38+1atX5/PPP+fss892HUlEREQcifUM8gpC64p7V2Js7/D1A7HuYMOGDa4jSJi1lvT0dAAGDRqUUM1xyfGZIrGmWhMvqM7EryKdQb4B+Cuh0+W6lxzZXM64XwFvALWBW6y1M6OQ1dc0g+wfL7/8Mr169eKUU05h7dq11KtXz3UkERERcSimM8jW2lmEtlarA8w3xrxtjBlgjOkW/rrLGDMPmBce824QmmPQDLJf5OXlce+99wLw+OOPJ1xzrNkW8YpqTbygOhO/imgGGcAYU4/QXsRXh58qe4GSM3znAz2ttYHY/0wzyP4wbNgwHnvsMVJSUvjwww+pUiXSVUQiIiKSaGK9Bhlr7V5r7a+BawidercJOBD+2hR+7hpr7TVBaY4BNm3a5DpC4H3zzTc8/fTTQGj/40RsjlNTU11HkIBQrYkXVGfiVxHPIEv5kpOT7cqVK13HCLTf/va3zJkzh169evHSSy+5jhMTeXl5JCUluY4hAaBaEy+ozsQrMZ9BlvJt377ddYRAe+edd5gzZw5169bliSeecB0nZsaMGeM6ggSEak28oDoTv1KDHCUNGzZ0HSGwioqKGDRoEBA6HOS0005znCh2evbs6TqCBIRqTbygOhO/qnasLzTGnA5cCDQF6vK/D+f9iLX24WO9T7zYu/eo56ZIDD3//PN89tlntGzZksGDB7uOE1OLFy9OqH2dxb9Ua+IF1Zn4VcQNsjGmKfA8/9vF4qjDCe1ykfANcu3atV1HCKSdO3cyfPhwIPRXdbVq1XKcKLbatGnjOoIEhGpNvKA6E7+KqEEOHx/9LnAmsAN4H7gOKAD+BpwCdAZOCH//H9EM62dFRUWuIwTSiBEj2LVrF7/85S/5zW9+4zpOzO3evdt1BAkI1Zp4QXUmfhXpDHIGcBawHLjKWrvbGFMM7LHW3gxgjKkDDAPuB4qstbdHM7BfaTcQ761evZrnnnuOqlWrMn78eIw54iqfhFFQUOA6ggSEak28oDoTv4q0Qb6W0JKJIdbacv/YZ63dBzxgjKkODDbG/Nta+/Jx5vS9GjVquI4QKNZa0tPTKS4u5q677uKnP/2p60ieaNmypesIEhCqNfGC6kz8KtJdLM4CigktrSitvO7wyfBjIGaQ8/PzXUcIlDlz5rB48WJOOukkRo4c6TqOZ5YtW+Y6ggSEak28oDoTv4q0Qa4G7LXWHir1XD5Qz5T5+21r7Q5gN9Du+CLGhwYNGriOEBgFBQXcfffdADzyyCOcdNJJjhN5p3v37q4jSECo1sQLqjPxq0gb5C1AA2NM6RnjzUBVoHXpgcaY2kADoM5xJYwTubm5riMExtixY8nOzqZdu3bccccdruN4avLkya4jSECo1sQLqjPxq4iOmjbGvA10Bc611n4Zfu5F4BZgorV2UKmxQ4HHgC+stQm/QLRjx452xYoVrmMkvM2bN9O6dWv27dvHv/71L7p06eI6kqeKioqoVu2Yty8XqTTVmnhBdSZeifVR0/8gtLdx6b8TeTb8ONAY8w9jzGPGmLnAo4Q+0DcjwnvEpY0bN7qOEAj33Xcf+/bt4/rrrw9ccwzQr18/1xEkIFRr4gXVmfhVpDPITYCJwFpr7Z9KPX8P8AShhtvyv1P13gB+b60tjlpin0pJSbFZWVmuYyS0//znP1x88cXUrFmTNWvW6NPPIiIiUikxnUG21m631vYo3RyHn38aaA88CEwFngautNb+LgjNMcCGDRtcR0hoxcXFpKenAzBkyJDANse9e/d2HUECQrUmXlCdiV9FNIMsR6YZ5Nh68cUXue222zjttNP46quvqFu3rutIIiIiEidiOoNsjFlpjFlhjDkz8miJLTs723WEhLVnzx6GDh0KwOjRowPdHPfp08d1BAkI1Zp4QXUmfhXpGuT9wEFrbb3YRYpP2sUidu655x7GjBnD//3f/7F06dJAHCl9JPrEt3hFtSZeUJ2JV2K9i8UW/vcBPCll69atriMkpK+++ooJEyZgjDn8GGTDhw93HUECQrUmXlCdiV9F2iD/P6COMeaCWISJZ40bN3YdISENHjyYoqIibr31Vjp27Og6jnN33nmn6wgSEKo18YLqTPwq0gb5UWAn8JwxplEM8sSt3bt3u46QcObNm8e8efOoV68ejz32mOs4vjBnzhzXESQgVGviBdWZ+FWkC39+AvwJGAN8ZYz5C/ABkAscOtKLrLVLjjlhnAjyB8di4eDBg2RkZAAwYsQITjnlFMeJ/KFz586uI0hAqNbEC6oz8atIG+R/EzoIBEJrkdPCX0djj+E+cefgwYOuIySUSZMmsXbtWlq1asXAgQNdx/GNDRs2cP7557uOIQGgWhMvqM7EryJtXDfxvwZZSgn6h8ei6dtvv2XkyJEAjBs3jho1ajhO5B+1a9d2HUECQrUmXlCdiV9F1CBba8+IUY64p21qoudPf/oTe/fu5eqrr+bqq692HcdXGjRo4DqCBIRqTbygOhO/ivRDenIEBQUFriMkhBUrVvDiiy9SrVo1xo4d6zqO73z55ZeuI0hAqNbEC6oz8Ss1yFFSr57OTjle1lrS09MPP7Zu3dp1JN/p0qWL6wgSEKo18YLqTPxKDXKU7Ny503WEuPfqq6/yn//8h5NPPlmbxx/BK6+84jqCBIRqTbygOhO/ivSo6W+O4R7WWnvWMbwuriQnJ9uVK1e6jhG38vPzad26NVu2bGHq1KncdtttriP5Ul5eHklJSa5jSACo1sQLqjPxSqyPmj7jGL8S3ubNm11HiGtPPPEEW7ZsoWPHjvTt29d1HN8aMmSI6wgSEKo18YLqTPwq0hnkSysYUh/oBPQDqhPaI3mLtfbdY04YJ1JSUmxWVpbrGHEpOzubNm3acODAAd577z3+7//+z3UkERERSSAxnUG21r5bwddca+1w4KdADqGjqT+N7C3Epw0bNriOELfuueceDhw4QM+ePdUcV6BXr16uI0hAqNbEC6oz8auIZpAjurAxnYH3gQnW2oyY3MRHNIN8bBYvXsxll11GnTp1+Oqrr2jWrJnrSCIiIpJgYr0GudKstcuAfOA3sbqHn2gGOXJFRUWkp6cDMHToUDXHlaDZFvGKak28oDoTv4rlDHJVIC98j1oxuYmPaAY5cpMnT2bAgAGcccYZfPHFFzpyVERERGLCNzPIwNVATSA3hvfwjZycHNcR4sp33313eK/jp59+Ws1xJWVkJPxqJfEJ1Zp4QXUmfhXVGWRjTA2gGXAdMAxoAEyx1qZG7SY+1aFDB/vxxx+7jhE3Bg4cyKRJk+jSpQvvvPMOxhjXkeJCbm4ujRs3dh1DAkC1Jl5QnYlXYjqDbIw5dLQvoABYBzwNnAhsAEZEco94tWPHDtcR4sZnn33Gs88+S5UqVRg/frya4whMmzbNdQQJCNWaeEF1Jn4V6RILU8mvjcBoIMVaG4glFvXr13cdIS5Yaxk0aBCHDh2if//+tG/f3nWkuNK1a1fXESQgVGviBdWZ+FW1CMe3rOD7RcBua23+MeaJW/v27XMdIS78/e9/55133uHEE0/k4Ycfdh0n7qxcuZJ27dq5jiEBoFoTL6jOxK8iapCttRtjFSTeVa9e3XUE39u/fz+DBw8G4OGHH6Zhw4aOE8WfU0891XUECQjVmnhBdSZ+FctdLER+YOzYsWzYsIFzzz2X/v37u44jIiIiUq6IZpCNMScBvwZ2WWvfqmDstYR2sZhrrd197BHjQ2FhoesIvrZlyxZGjRoFwIQJE6hWLdLVPQKwbds21xEkIFRr4gXVmfhVpDPINwPTgORKjL0kPDYQx+TUqVPHdQRfu//++8nPz6d79+788pe/dB0nbiUnV+ZfPZHjp1oTL6jOxK8ibZC7hx9nV2LsDEI7Wvw2wnvEpT179riO4FsffPABM2fOpGbNmjz99NOu48S1+fPnu44gAaFaEy+ozsSvIm2QzwKKgbWVGPtleOxPIg0Vjxo1auQ6gi8VFxeTnp4OwN13382ZZ57pOFF869u3r+sIEhCqNfGC6kz8KtIGuRGw11pbVNFAa20hsAc4+ViCxRutoyrfjBkz+Oijj2jatClDhw51HSfulazjFok11Zp4QXUmfhXRUdPGmG1AY+Aka+3eCsbWB3YCO621pxxXyjiQkpJis7KyXMfwlb1799KqVSu+/fZbXnrpJXr1CsRydBEREfGZmB41DawgtK64dyXG9g5f/+MI7xGXNmzY4DqC7zz66KN8++23XHjhhdx0002u4yQE/SFDvKJaEy+ozsSvIp1BvgH4K7AP6G6tXXSEcb8C3gBqA7dYa2dGIauvaQb5h9atW8e5555LYWEhy5cvp1OnTq4jiYiISEDFdAbZWjsLeAeoA8w3xrxtjBlgjOkW/rrLGDMPmBce824QmmPQDHJZgwcPprCwkL59+6o5jiLNtohXVGviBdWZ+FVEM8gAxph6wCvA1eGnyl7AhB/nAz2ttYHY/0wzyP+zYMECunbtygknnMDatWtp0qSJ60giIiISYLFeg4y1dq+19tfANcAsYBNwIPy1KfzcNdbaa4LSHANs2rTJdQRfKCwsJCMjA4Dhw4erOY6y1NRU1xEkIFRr4gXVmfhVxDPIUr7k5GS7cuVK1zGcGzduHIMHD+bss8/ms88+o0aNGq4jJZS8vDySkpJcx5AAUK2JF1Rn4pWYzyBL+bZv3+46gnP//e9/GTlyJABjx45VcxwDY8aMcR1BAkK1Jl5QnYlfVYtksDGmKnAaUGSt3VrB2Kbh62+21hYfe8T40LBhQ9cRnBs2bBh79uzhqquu4pprrnEdJyH17NnTdQQJCNWaeEF1Jn4V6QzyDcAG4OFKjB0THnt9pKHi0d69Rz03JeGtWrWKqVOnUq1aNcaNG4cxpuIXScQWL17sOoIEhGpNvKA6E786lgYZ4MVKjH2e0I4WN0Z4j7hUu3Zt1xGcsdaSlpaGtZaBAwfSpk0b15ESlv7ZildUa+IF1Zn4VaQN8k/Dj59UYuxH4cf2Ed4jLhUVFbmO4MysWbN47733aNy4MSNGjHAdJ6Ht3r3bdQQJCNWaeEF1Jn4VaYPcFNhtrc2vaGB4zO7waxJeUHcD2bdvH0OGDAHgscceo0GDBo4TJbaCggLXESQgVGviBdWZ+FWkDfI+IMkYU+GH+4wx1YG6wMFjCRZvgrpjw5NPPsnmzZs577zzuPXWW13HSXgtW7Z0HUECQrUmXlCdiV9F2iB/SWhniqsqMfYqoDqwNtJQ8Sg/v8JJ9YSzceNGRo8eDUBmZiZVq1Z1nCjxLVu2zHUECQjVmnhBdSZ+FWmDPIfQB+/GGWOOeESaMeZUYDyhY6jfPPZ48SOISwuGDBnC/v37ufHGG7n44otdxwmE7t27u44gAaFaEy+ozsSvIm2QJwMbgTOBT4wx9xhjzjXGnBD++qkx5l7gY6AlsBmYGGkoY0wVY0yGMeZLY8x+Y0yOMWaMMaZuJV77kDHGHuWrsJzXtDbGvGmM2WWMyTfGLDXGXBZJ5tzc3EiGx713332X2bNnU7t27cOzyBJ7kydPdh1BAkK1Jl5QnYlfRXzUtDGmLbAAaEZohrjcYcAWoKu19rOIQxkzAUgjNGM9HzgHGAgsBS4/2sEjxpj2lL9zRntgCDDHWvvbUuPPApYDRYRmvfcAtxPasaOrtfaflcncsWNHu2LFisoMjXuHDh2iY8eOfPLJJ4wcOVI7V3ioqKiIatUiOt9H5Jio1sQLqjPxSsyPmrbWfgH8DHga+JZQM1z661vgKeBnx9gcn0uoGX7DWvtba+2frbWDgcFAFyrYV9la+6m1dmbZL6KljFIAACAASURBVKB+eMgLZV7yONAAuNJa+7i1djLwc2Ar8Iyp5IkXGzdurPR7jHd//vOf+eSTT2jRosXhHSzEG/369XMdQQJCtSZeUJ2JX0U8g/yjCxhzOnAKoeZ4u7V2Y5nv17PWVvqYOWPMo8CfgEustUtLPV8L2Am8a629OsKMdYBtwPfA6dbaQ+Hn64av+R9r7S/LvGY4oRMDL7DWLq/oHikpKTYrKyuSWHFp165dnH322ezcuZPXXnuNHj16uI4kIiIiclQxn0Euy1q70Vq73Fr7YUlzbEKuMsb8lVBjGolOQDGhZQ+l77Of0NrmTscQ8/dAPWBaSXMc1h6oCXxQzmtKPlpbqftt2LDhGGLFn4ceeoidO3dy6aWX8rvf/c51nMDp3bu36wgSEKo18YLqTPzquBvk0sIf2BtN6MN5/yB0NHWtCC/TFNhhrT1Qzve2AI2MMZFuOnwbofXSZY/ILjnEZMsR7gVw2pEuaoy5wxiTZYzJAliyZAlz585l1qxZLF++nMzMTHJychg6dChFRUX06dMH+N8vhD59+lBUVMTQoUPJyckhMzOT5cuXM2vWLObOncuSJUuYMmUK69atY+TIkeTl5ZGamgpAr169fvCYkZFBbm4uo0ePZvXq1cyYMYOFCxeycOFCZsyYwerVqxk9ejS5ublkZGSUe43U1FTy8vIYOXIk69atY8qUKT94T6+++ioTJ07EGMNZZ53FoUOH4v49xdvP6aWXXkq495SIP6dEeE8vvfRSwr2nRPw5xft7uvzyyxPuPSXizykR3lOkorHEoiHQE7gFOK/kaaAQ+BfwN2vt1Aiutx6obq1tUc73/gL0Bk601lbqfEpjTGtC+ze/Y629vMz3egN/AW6z1r5Y5ntnAuuBCdbaQRXdp1GjRnbHjh2ViRSXrLVceeWVLFq0iP79+/Pss8+6jhRIffr0Yfr06a5jSACo1sQLqjPxSqRLLI7po6Phk/S6EWqKu4avYwjN0r4NzAbestbuOYbL7wNOPsL3apUaU1m3hR/La9JLrlPzeO91+umnRxAp/sydO5dFixbRoEEDHnnkEddxAmvq1Er/WVPkuKjWxAuqM/GriJZYGGNSjDETCa0rfh24llBzvLTUsN7hnSOOpTmG0O4RjYwx5TWtpxFaflGp46vDjfzNwHeEtowr714l1y3vXlD+8osfX2jr1ooHxakDBw4wePBgAEaOHEmjRo0cJwqu4cOHu44gAaFaEy+ozsSvKmyQjTFNjTH3GmM+Bz4EBgANgc+A+4EzrLW/iGKmj8K5zi+ToxbQAYhkq4huhHbYeOkIa5pXAweAC8v5XufwY6Xu17hx4whixZdx48bxzTff0LZt28PrfMSNO++803UECQjVmnhBdSZ+ddQG2RizgNDJeY8TOqwjB3gSaG+t/Zm1drS1NifKmWYRWqpRdt3v7UAd4OVS+c4yxrQ5yrVKlleU3fsYAGttHvAW8AtjzM9KXTcJ6Aeso8xuGkeye3ellkTHna1bt/Loo48CMGHCBP5/e/cep1VZ7n/8cymMIAgkmOIhZXtASwnHSSnLpNyWZtus7OCGxCQLUxD8gWKi4gEJREQNt7YVj/TDdtLGDHWXmJYHHJDAgsKfHIaA7UABjozAyPX7Y62xx/GZE6xZ95pnfd+v1/N6nHX8PsNVXN6u5747duwYOFG+zZ5d7D+EiCRPtSZpUJ1JVjX3DPLpRM3qTOBud/99Wwdy9yVm9hPgEjN7DPg1UXM+HPhdnKXeb4FDiZ5/fh8zOxD4IjDf3Zc0ccuxwOeBp81sKrCFqBk/CPiSt/BbjF26NLsKdrs0duxY3n77bc4++2xOO+205k+QNjVgwIDmDxJJgGpN0qA6k6xq6TPIXwG+H89tvGdbBopdBvwf4GPAT4hWz7sDOKupZaYbGALsSfEv573H3V8HTiaa9/hKohUC3wa+6O5PtTTw9u0teiy6XXn55Zd58MEHKSsrY8qUKaHjCPmZb1vCU61JGlRnklXNjSCfQzRTxZeAfyeazm2jmc0CfubuL7RFqHgxjynxq6njDmti3wRgQgvvtxQ4uxURP6CFK1K3Gzt37mT48OEAjBo1isMPPzxwIgHo3Llz6AiSE6o1SYPqTLKqyRFkd/9vd/8q0BsYAbwK9CL6ot7zZvaGmd1oZh9r+6jZ1qHDLs2Yl1kPPfQQ8+fPp3fv3lx11VWh40isR48eoSNITqjWJA2qM8mqFj1i4e5/d/c74gmWjyV6DGE9cBjRM7yLCw7/wAIfeVBbWxs6QmLeeustrrzySgAmTpzIPvvsEziR1Fu2bFnoCJITqjVJg+pMsqrVS027+5/dfQxwCHAm0aIg9VOoGfBHM1toZleb2THJRc22bt26hY6QmJtuuon169dz0kknvbdco2TDwIEDQ0eQnFCtSRpUZ5JVrW6Q67n7Tnd/0t2/BRwA/AB4gahJ7g+MB14zsz8nkjTjNm7cGDpCIl5//XWmTp0KwO23384ee+xyiUgbmDlzZvMHiSRAtSZpUJ1JVlkLZzFr+QXNDieaQWIQ0RRs7u5pzHwRVHl5uS9cuDB0jN129tlnM2fOHM4//3zuv//+0HGkgZqaGrp27Ro6huSAak3SoDqTtJjZgvhR4RZJfHjQ3f+fu49z9z5E8ws/mPQ9smjNmjWhI+y2p59+mjlz5tC1a1duvvnm0HGkiNGjR4eOIDmhWpM0qM4kqxIfQc6riooKr6xszSrY2bJjxw4+/vGPs3TpUiZOnMgVV1wROpKIiIhIIoKPIOdVe5/sfPr06SxdupQjjjiCyy5ruMq3ZIW+NClpUa1JGlRnklUaQU5Iex5Brq6u5sgjj2Tz5s3MmTOHL3/5y6EjiYiIiCRGI8iBtOcR5HHjxrF582ZOP/10zjrrrNBxpAkabZG0qNYkDaozySqNICekvY4gL1q0iPLycvbYYw+WLFnCMcfkZupqERERyQmNIAdSVVUVOkKruTsjRozA3bnkkkvUHLcDI0eODB1BckK1JmlQnUlWaQQ5If379/dFixaFjtEqjz76KN/85jfp1asXy5cvp0ePHqEjSTOqq6vZb7/9QseQHFCtSRpUZ5IWjSAHsmHDhtARWmXr1q3vzT950003qTluJ2bMmBE6guSEak3SoDqTrFKDnJDu3buHjtAqkydPZvXq1fTv358LL7wwdBxpoTPOOCN0BMkJ1ZqkQXUmWaUGOSFbt24NHaHFVq9ezY9//GMApk2bxp57lvxK4CWjFJYzl/ZBtSZpUJ1JVqlBTkjHjh1DR2ixMWPGUFtbyze+8Q1OOeWU0HGkFXr37h06guSEak3SoDqTrFKDnDPPPfccs2bNolOnTkyePDl0HBEREZHMUYOckB07doSO0Kx3332XESNGAHDFFVfwkY98JHAiaa1169aFjiA5oVqTNKjOJKvUICdk7733Dh2hWffeey+LFi3ikEMOYcyYMaHjyC4oLy8PHUFyQrUmaVCdSVapQU7I5s2bQ0do0qZNm/jRj34EwC233NIuGnr5oLlz54aOIDmhWpM0qM4kq9QgJ6RXr16hIzRp/PjxbNiwgVNOOYVzzz03dBzZRRdccEHoCJITqjVJg+pMskoNckKy/BzV0qVLufPOOzEzpk2bhpmFjiS7aMKECaEjSE6o1iQNqjPJKi01nZCKigqvrKwMHeMD3J0zzjiDp556iosuuoi77747dCQRERGRVGmp6UBWrFgROkJRv/rVr3jqqafo3r07N954Y+g4spsGDRoUOoLkhGpN0qA6k6zSCHJCsjiCvG3bNo499lhef/11brvttvemeBMRERHJE40gB5LFEeRp06bx+uuvc8wxx3DxxReHjiMJ0GiLpEW1JmlQnUlWaQQ5IVkbQV6/fj1HHnkkNTU1PPnkk3zhC18IHUlEREQkCI0gB7J69erQEd5n7Nix1NTU8OUvf1nNcQkZNmxY6AiSE6o1SYPqTLJKI8gJKS8v94ULF4aOAcD8+fM56aSTKCsr409/+hNHHHFE6EiSkJqaGrp27Ro6huSAak3SoDqTtGgEOZD169eHjgDAzp07GT58OAAjR45Uc1xipkyZEjqC5IRqTdKgOpOsUoOckJ49e4aOAMAjjzzCyy+/zAEHHPDe0tJSOs4777zQESQnVGuSBtWZZJUa5IRs2bIldARqamq44oorAJg4cSL77LNP4ESStHnz5oWOIDmhWpM0qM4kq9QgJ6Rz586hIzBhwgTWrVvHiSeeyODBg0PHkTZw9NFHh44gOaFakzSoziSr1CAnpK6uLuj933jjjfee5Zo2bRp77KE/2lK0adOm0BEkJ1RrkgbVmWSVuqiEhJ4N5PLLL2f79u0MHjyYAQMGBM0ibae2tjZ0BMkJ1ZqkQXUmWaUGOSFlZWXB7v2b3/yGX/7yl3Tp0oWJEycGyyFtr0+fPqEjSE6o1iQNqjPJKjXICXn77beD3Leuro4RI0YAcPXVV3PggQcGySHpeOmll0JHkJxQrUkaVGeSVWqQE9KjR48g973rrrv485//zL/8y79w2WWXBckg6TnnnHNCR5CcUK1JGlRnklVqkBNSXV2d+j03bNjANddcA8Ctt95Kp06dUs8g6Zo+fXroCJITqjVJg+pMskpLTSfkhBNO8AULFqR6z4svvpi77rqL0047jaeffhozS/X+kr66ujo6dOgQOobkgGpN0qA6k7RoqelAVq1aler9/vjHP3L33Xez5557Mm3aNDXHOTF06NDQESQnVGuSBtWZZJVGkBNSUVHhlZWVqdzL3fnc5z7Hs88+y/Dhw5k2bVoq9xURERFpjzSCHMiKFStSu9cvfvELnn32WXr27Ml1112X2n0lPK2QKGlRrUkaVGeSVRpBTkhaI8i1tbUcc8wxrFq1irvuuosf/OAHbX5PERERkfZMI8iBrFy5MpX73HLLLaxatYp+/frxve99L5V7SnYMGTIkdATJCdWapEF1JlmlEeSEpDGLRVVVFX379qW2tpZ58+Zx6qmntun9JHv0jW9Ji2pN0qA6k7RoBDmQtWvXtvk9rrjiCmpra/n617+u5jinxo0bFzqC5IRqTdKgOpOs0ghyQvr16+eLFy9us+v//ve/5zOf+QydOnVi2bJlHHrooW12L8muqqoqDjnkkNAxJAdUa5IG1ZmkRSPIgWzatKnNrv3uu+8yfPhwAMaMGaPmOMdmz54dOoLkhGpN0qA6k6xSg5yQLl26tNm1Z8yYwauvvsrBBx/MmDFj2uw+kn0DBgwIHUFyQrUmaVCdSVapQU7I9u3b2+S6mzdv5qqrrgJg8uTJbdqIS/alOd+25JtqTdKgOpOsUoOckLZa6vn666+nurqaT3/603zzm99sk3tI+9G5c+fQESQnVGuSBtWZZJUa5IS0xTQ1y5Yt4/bbb8fM3nuXfOvRo0foCJITqjVJg+pMskoNckJqa2sTv+aoUaOoq6tj6NChHH/88YlfX9qfZcuWhY4gOaFakzSoziSr1CAnpFu3bole74knnmDu3Ll0796dG2+8MdFrS/s1cODA0BEkJ1RrkgbVmWSVGuSEbNy4MbFrbd++nZEjRwJw7bXX8uEPfzixa0v7NnPmzNARJCdUa5IG1ZlklRYKSUh5ebkvXLgwkWvdcsstjB49mr59+7J48WLKysoSua60fzU1NXTt2jV0DMkB1ZqkQXUmadFCIYGsWbMmkeusX7+e66+/HoDbbrtNzbG8z+jRo0NHkJxQrUkaVGeSVRpBTkhFRYVXVlbu9nUuvPBC7rvvPs466ywef/zxBJKJiIiI5JtGkANJYrLzyspKZsyYQceOHbn11lsTSCWlZtCgQaEjSE6o1iQNqjPJKo0gJ2R3R5DdnZNPPpkXX3yR0aNHM2nSpATTiYiIiOSXRpAD2d0R5JkzZ/Liiy+y//77c/XVVyeUSkqNRlskLao1SYPqTLJKI8gJ2Z0R5JqaGvr27cvatWu57777uOCCCxJOJyIiIpJfGkEOpKqqapfPnThxImvXrqWiooLzzz8/wVRSaurnxxZpa6o1SYPqTLJKI8gJ6d+/vy9atKjV573xxht89KMfZdu2bbzwwgt88pOfbIN0Uiqqq6vZb7/9QseQHFCtSRpUZ5IWjSAHsmHDhl06b/To0Wzbto1BgwapOZZmzZgxI3QEyQnVmqRBdSZZpQY5Id27d2/1Oc888wyPPfYYXbp0YeLEiW2QSkrNGWecETqC5IRqTdKgOpOsUoOckK1bt7bq+Lq6OkaMGAHAVVddxUEHHdQWsaTEJLWcuUhzVGuSBtWZZJUa5IR07NixVcfffffdvPbaa/Tp04dRo0a1USopNb179w4dQXJCtSZpUJ1JVmWyQTazPcxspJktM7N3zKzKzKaYWZdWXGNfM7vFzF6Pr1FtZvPM7DNFjj3JzH5jZm+Z2RYze9LM+if7qf5p48aNjBs3DoApU6bQqVOntrqViIiIiLRSh9ABGjEVGA7MBqYAx8Q/H29mp7n7zqZONrNDgWeBrsC9wF+B7kA/4KAGxw6Ij/0bcE28+RLgeTP7lLsvaUngHTt2tOQwAK699lr+8Y9/8PnPf56vfOUrLT5PZN26daEjSE6o1iQNqjPJqsw1yGb2MeBS4DF3/1rB9hXA7cC3gJnNXOZhos/Wz92b+1/f7cB24BR3/1t8r0eBpUTN+ektyb333nu35DCWLFnCXXfdxZ577sltt92GmbXoPBGA8vLy0BEkJ1RrkgbVmWRVFh+x+DZgwG0Ntv8U2Ao0uS6lmZ0CfBqY5O7rzKyjmRXtXs3sCOATwM/rm2OA+J9/DpxmZge0JPTmzZubPcbdGTFiBDt37mTYsGEce+yxLbm0yHvmzp0bOoLkhGpN0qA6k6zKYoP8CWAnML9wo7u/AyyK9zflzPh9tZk9DtQCb5vZX82sYXNdf60Xi1znJaJG/YSWhO7Vq1ezx8yePZt58+ax7777Mn78+JZcVuR9tAy5pEW1JmlQnUlWZbFBPhDY4O7biuz7G9DLzMqaOL9v/P5TYF/gfOBCoscoHjKzwv81Hlhw3WL3ggbPLDemueeo3nnnHS6//HIAbrjhBvbdd9+WXFbkfSZMmBA6guSEak3SoDqTrMpig7w3UKw5Bnin4JjG7BO/vwUMdPdH3P0+4DPAJmCCmdV/7vrrFLtfs/cys4vMrNLMKuvq6njuueeYM2cOs2bNYv78+dx+++1UVVUxduxYJk2axMqVKznuuOP4wx/+AMCQIUOoq6tj7NixVFVVcfvttzN//nxmzZrFnDlzeO6557jnnntYvnw548ePp6amhmHDhgEwaNCg972PHDmS6upqJk2axJIlS3jggQd4+umnefrpp3nggQdYsmQJkyZNorq6mpEjRxa9xrBhw6ipqWH8+PEsX76ce+65p8nPVFdXx5AhQwAYPHiwPlMKn2nq1Kkl95lK8c+pFD7T1KlTS+4zleKfU3v/TP379y+5z1SKf06l8Jlay9y91Se1JTNbAnzY3fcvsu9R4FxgL3ff3sj5jwNnATe5+9UN9j0AfAf4qLsvNbPLgVuAM919boNjzwSeAL7v7vc0l7tnz56+cePGovvWrFlD37592bp1K8888wwDBw5s7nIiRQ0aNIiHH344dAzJAdWapEF1JmkxswXuXtHS47M4gryW6DGKvYrsO4jo8YuizXFsTfy+vsi++ucgPlRwr/rrFrsXFH/84gP69OnT6L4rr7ySrVu38rWvfU3NsewW/UUiaVGtSRpUZ5JVWWyQXyHKdWLhRjPrBPQHKps5v/7LfQcX2Ve/7c2CewF8ssixAwAHFjRzPwBWrFhRdPsLL7zAI488wl577cXkyZNbcimRRtX/ZyORtqZakzSoziSrsviIxXHAH4HZDeZBvpRozuLB7v5wvO1woKO7Lys47kPAKmALcLS718TbewPLgbXuflTB8a8QfbHvaHdfG287EFgGzHf301qSu6Kiwisr39+779y5kxNPPJEFCxZw9dVXc8MNN7TytyEiIiIiu6vdP2IRr1z3E+CrZvaYmQ01synArcDveP8iIb8lWtCj8Px/AP+H6BGJl8xslJldSTRtWxnRKnmFRgB7Ea2cd5mZXQY8T/S7ubyluVevXv2Bbffffz8LFizgoIMO4sorr2zppUQaVf9FBJG2plqTNKjOJKsyN4IMYGZ7ApcBFwGHARuAWcA19SPC8XErgUPd/QPL0ZnZV4ExwHFE8yq/CIx39z8UOfaTwI3ASUSPVbwAjHX3hS3NXF5e7gsX/vPwzZs3c9RRR/Hmm2/yyCOPcN5557X0UiKNqqmpoWvXrqFjSA6o1iQNqjNJS7sfQQZw93fdfYq793X3vdz9IHcfVdgcx8cdVqw5jvc95u4D3L2Lu+/j7qcXa47jY19098+7e9f42C+0pjkGWL/+/d8JvPHGG3nzzTc5+eST+fa3v92aS4k0asqUKaEjSE6o1iQNqjPJqkyOILdHxx13nC9ZsgSAv/71rxx77LHU1dXxyiuvcMIJLVqMT6RZy5cv58gjjwwdQ3JAtSZpUJ1JWkpiBLk92rJly3v/PGrUKHbs2MF3v/tdNceSqHnz5oWOIDmhWpM0qM4kq9QgJ6Rz584AzJ07lyeeeIJu3bpx0003BU4lpeboo48OHUFyQrUmaVCdSVapQU5IXV0d27dvf2/5xWuuuYb99//AYoAiu2XTpk2hI0hOqNYkDaozySo1yAlxd+68807+8pe/cNRRR3HppZeGjiQlqLa2NnQEyQnVmqRBdSZZpQY5IXvssQfjx48HYOrUqZSVlQVOJKWoqSXNRZKkWpM0qM4kq9QgJ2TdunVs2bKFM888kzPPPDN0HClRL730UugIkhOqNUmD6kyyStO8JcTMvEOHDrz22mv07ds3dBwpUVVVVRxyyCGhY0gOqNYkDaozSYumeQtoxIgRao6lTU2fPj10BMkJ1ZqkQXUmWaUR5IR07NjRN2zYQPfu3UNHkRJWV1dHhw4dQseQHFCtSRpUZ5IWjSAH0rlzZzXH0uaGDh0aOoLkhGpN0qA6k6zSCHJCKioqvLKyMnQMEREREWlAI8iBrFixInQEyYHBgweHjiA5oVqTNKjOJKs0gpwQjSCLiIiIZJNGkANZuXJl6AiSA0OGDAkdQXJCtSZpUJ1JVmkEOSEnnHCCL1iwIHQMKXH6xrekRbUmaVCdSVo0ghzI2rVrQ0eQHBg3blzoCJITqjVJg+pMskojyAnp16+fL168OHQMKXFadUrSolqTNKjOJC0aQQ5k06ZNoSNIDsyePTt0BMkJ1ZqkQXUmWaUGOSFdunQJHUFyYMCAAaEjSE6o1iQNqjPJKjXICdm+fXvoCJIDmm9b0qJakzSoziSr1CAnxMxCR5Ac6Ny5c+gIkhOqNUmD6kyySg1yQjRNjaShR48eoSNITqjWJA2qM8kqNcgJqa2tDR1BcmDZsmWhI0hOqNYkDaozySo1yAnp1q1b6AiSAwMHDgwdQXJCtSZpUJ1JVqlBTsjGjRtDR5AcmDlzZugIkhOqNUmD6kyySguFJKS8vNwXLlwYOoaUuJqaGrp27Ro6huSAak3SoDqTtGihkEDWrFkTOoLkwOjRo0NHkJxQrUkaVGeSVRpBTkhFRYVXVlaGjiEiIiIiDWgEORBNdi5pGDRoUOgIkhOqNUmD6kyySiPICdEIsoiIiEg2aQQ5EI0gSxo02iJpUa1JGlRnklUaQU6IRpBFREREskkjyIFUVVWFjiA5MHLkyNARJCdUa5IG1ZlklUaQE9K/f39ftGhR6BhS4qqrq9lvv/1Cx5AcUK1JGlRnkhaNIAeyYcOG0BEkB2bMmBE6guSEak3SoDqTrFKDnJDu3buHjiA5cMYZZ4SOIDmhWpM0qM4kq9QgJ2Tr1q2hI0gOaDlzSYtqTdKgOpOsUoOckI4dO4aOIDnQu3fv0BEkJ1RrkgbVmWSVGmQRERERkQJqkBOyY8eO0BEkB9atWxc6guSEak3SoDqTrFKDnJC99947dATJgfLy8tARJCdUa5IG1ZlklRrkhGzevDl0BMmBuXPnho4gOaFakzSoziSrtFBIQrRQiKRBk+pLWlRrkgbVmaRFC4UEoueoJA0TJkwIHUFyQrUmaVCdSVZpBDkhFRUVXllZGTqGiIiIiDSgEeRAVqxYETqC5MCgQYNCR5CcUK1JGlRnklUaQU6IRpBFREREskkjyIFoBFnSoNEWSYtqTdKgOpOs0ghyQjSCLCIiIpJNGkEOZPXq1aEjSA4MGzYsdATJCdWapEF1JlmlEeSElJeX+8KFC0PHkBJXU1ND165dQ8eQHFCtSRpUZ5IWjSAHsn79+tARJAemTJkSOoLkhGpN0qA6k6xSg5yQnj17ho4gOXDeeeeFjiA5oVqTNKjOJKvUICdky5YtoSNIDsybNy90BMkJ1ZqkQXUmWaUGOSGdO3cOHUFy4Oijjw4dQXJCtSZpUJ1JVqlBTkhdXV3oCJIDmzZtCh1BckK1JmlQnUlWqUFOiGYDkTTU1taGjiA5oVqTNKjOJKvUICekrKwsdATJgT59+oSOIDmhWpM0qM4kq9QgJ+Ttt98OHUFy4KWXXgodQXJCtSZpUJ1JVqlBTkiPHj1CR5AcOOecc0JHkJxQrUkaVGeSVWqQE1JdXR06guTA9OnTQ0eQnFCtSRpUZ5JVWmo6ISeccIIvWLAgdAwpcXV1dXTo0CF0DMkB1ZqkQXUmadFS04GsWrUqdATJgaFDh4aOIDmhWpM0qM4kqzSCnJCKigqvrKwMHUNEREREGtAIciArVqwIHUFyYPDgwaEjSE6o1iQNqjPJKo0gJ0QjyCIiIiLZpBHkQFauXBk6guTAkCFDQkeQnFCtSRpUZ5JVGkFOiGaxkDToG9+SFtWapEF1JmnRCHIga9euDR1BcmDcuHGhI0hOqNYkDaozySqNICekX79+vnjx4tAxpMRVVVVxyCGHhI4hOaBakzSoziQtJKBvOwAAD4JJREFUJTGCbGZ7mNlIM1tmZu+YWZWZTTGzLi083xt51TRyfF8z+6WZ/cPM3jaz583sc63JvGnTptYcLrJLZs+eHTqC5IRqTdKgOpOsyuqDP1OB4cBsYApwTPzz8WZ2mrvvbME1ngfuabBtR8ODzOxw4AWgDpgEbAa+BzxlZme4+29aErhLlxb17iK7ZcCAAaEjSE6o1iQNqjPJqsw1yGb2MeBS4DF3/1rB9hXA7cC3gJktuNQb7v5wC467GegBnODui+J7PQj8CfiJmR3tLXgOZfv27S24lcjuWbFiBSeeeGLoGJIDqjVJg+pMsiqLj1h8GzDgtgbbfwpsBQa19EJmVmZmXZvY3wX4N+DZ+uYYwN1rgP8EjgI+0cJ7tTSWyC7r3Llz6AiSE6o1SYPqTLIqiw3yJ4CdwPzCje7+DrCIFjaswNeJGuq3zOxNM7vDzLo3OKYfsBfwYpHzXyrI0yxNUyNp6NGjR+gIkhOqNUmD6kyyKotd3YHABnffVmTf34BPmVmZuzf1TMN84OfA60A34EzgEuCzZvapeIS4/l711y12L4CDGruJmV0EXBT/uM3MXmsik0gSegEbQoeQXFCtSRpUZ5KWvq05OIsN8t5AseYY4J2CYxptkN39pAabHjSzxcBNwIj4vf46NHK/dxocU+w+9xB/EdDMKlszfYjIrlCdSVpUa5IG1ZmkxcwqW3N8Fh+x2Er02EMxnQqOaa3JRE31lxrci0butzv3EhEREZF2KosN8lqgl5kVa1oPInr8otVTRrj7jvprN7hX/XWL3QuKP34hIiIiIiUqiw3yK0S53jfvi5l1AvoDrRoib3D+wcD/FmxeQvR4xSeLnFI/OWNL79dwzmWRtqA6k7So1iQNqjNJS6tqLYsN8izAgcsabP8e0fPAj9RvMLPDzezowoPMrGcj172B6Jnrx+s3xF/Wexw41cw+XnCNrsBQYDkNZtNoTPw8skibUp1JWlRrkgbVmaSltbVmLVgDI3VmdgfRrBOzgV/zz5X0/gB8rn4lPTNbCRzq7lZw7lSi0d95wGqgK9EsFgOBl4GB7l5bcPwRRE3wDqIV/LYQNePHAV9y96fa8rOKiIiISLZktUHek2gE+SLgMKIpYGYB1xRM0dZYg3w2cDFwLNATeJdoJPhR4NZ4PuWG9zsGmAh8FigDFgLXtXSZaREREREpHZlskEVEREREQsniM8jtgpmNNbOfm9kbZubxaLZIoszsKDO73sxeMrNqM3vLzBaZ2Y/ipdJFdpuZ9TWzR8xsqZltNrOtZrbMzG41s96h80lpM7O9zWxF/HfpnaHzSOmIa6rYq6a5c7O4UEh7MQH4O9HjGForU9rKd4EfAnOIvqC6g+h5+huBb5jZgMJn6kV20cFAb6LvfawB6oi+h3ER8C0z6+/ubwbMJ6Xtet4/BatIkp7ngzNY7GjuJDXIu+5wd38DIF5iumvgPFKa/gu42d03F2z7DzNbDvwIuBDQiIvsFnf/LfDbhtvN7Dmi728MASalHEtywMzKib5zNAaYEjiOlKY33P3h1p6kRyx2UX1zLNKW3L2yQXNcb1b8fmyaeSR3VsXvHwqaQkpS/IX8nwJPAo8FjiMlzMzK4il8W0wNskj7dHD8/r9NHiXSCmbWycx6mdnBZnY6cHe869chc0nJGgkcTTStq0hb+TqwFXjLzN40szvMrHtzJ+kRC5F2Jh51uYboOdGZgeNIaRkK3FHw80pgkLs/HyaOlCoz6wOMB65395VmdljYRFKi5gM/B14HuhGti3EJ8Fkz+1Th1MENqUEWaX9uI1oM5yp3/0voMFJSfgksI/pOxfHAvwH7BU0kpeouYAVwa+ggUrrc/aQGmx40s8XATcCI+L0oNcgi7YiZ3UD0b7/3uPvNofNIaXH3NUSzWAD80sx+AbxiZp1Vb5IUMxsEnA6c4u7NziYgkrDJwLXAl2iiQdYzyCLthJldB1wNzAB+EDaN5IG7LwZeJVqdVGS3mdleRKPGvwbWm9kRZnYEcGh8SPd4m6ZPlTYR/0vZWpqZWlANskg7YGbXEv0b74PAUNcSmJKezsC+oUNIyehM9NjOl4DlBa9n4/2D4p+Hhggnpc/MOhF90b3JL7nrEQuRjDOza4DrgIeAC9x9Z9hEUmrM7AB3X19k+0CiqQSfTT2UlKq3gXOLbN8PmE405du9wOI0Q0npMbOe7r6xyK4biPrfx5s8XwNRu8bMBvPP/yR0KVDGPyc5X+XuDwUJJiXFzH5ItBDIamAc0LA5/l93/5/Ug0lJMbPZRCvpPUM093En4ATgW0TTI53q7ovCJZRSF89isQL4ibtr2jfZbWY2legL7fOI/g7tSjSLxUDgZWBgUyvRagR5110IfLbBthvi998RjfaJ7K5PxO8fAR4osv93gBpk2V0/A84HBhON5DlRo3w3MNndVwfMJiKyK54FPkr0/209gXeJHt/5EXCru7/T1MkaQRYRERERKaAv6YmIiIiIFFCDLCIiIiJSQA2yiIiIiEgBNcgiIiIiIgXUIIuIiIiIFFCDLCIiIiJSQA2yiIiIiEgBNcgiItIqZvasmbmZDQmdpTXMbGWc+9TQWUQk27SSnoiItGvxMsVDgE3uflvQMCJSEjSCLCIi7d1hwLXAZYFziEiJUIMsIiIiIlJADbKIiIiISAE1yCIiDRR+mcvMepvZf5hZlZnVmtlSMxtpZnsUHH+umT1vZpvMbIuZPWFmxxa5bpmZfcnMfmpmfzSzDWb2jpmtMrNHzOyERvLcHOepNrMDGjnmyfiYBWbWMYHfwRfN7Bkz2xx/ppfMbHALzy0zs0vi38nfzWxb/BnvM7NjGjnn/jj/dWbWyczGm9my+Hf+ppn9zMyOKnLeSmBe/OOh8TUKX0Maud++Znarma2I8/0t/nPp3bLfkIiUMjXIIiKN6wMsBL4PdAM6AkcDtwLTAMxsIvAo8Emi/0/dBzgTeN7MjmxwvdOBXwFDgX5AZ8CBjwDnAY01odfEOXoB9zXcaWaXAF8AaoFB7r5jlz9xdL3RwFxgYPx53gU+ATxoZlOaObc3MB+4A/g00B3YRvQZLwAWmtlXm7jEXkQN7zVEv//twH7At4BXzeyUBsdXA/+I/3kn8L8NXrVF7nEw0e9zJPBhoj+DA4n+XF4wsw819RlFpPSpQRYRadxUYAXwcXfvTtQkj4v3/dDMrgJGEX05rLu7dwOOA/4C9ABuanC9GmAG8Hmgl7t3cffOwKHAbUQzC91jZh8pPClueAcRNXtnmNnF9fvMrC8wKf7xCndfujsf2Mw+Dfw4/vFh4EB3/xDQM77PKKB/I+d2BP4b+DjwHHAK0Dn+vRwATAE6AQ+Z2eGNRBhG9C8P5wNd49/78UQN7d7Ao4UNrLt/AqhvuKvc/YAGr1lF7nEHUVP9KXfvAnQFzgY2EX3hb2wTvyIRyQN310svvfTSq+AFrCQaVfw70KPI/t/G+x24psj+z8T73gHKWnHfe+Pzrm1k/6Xx/q1AX6KG+pV429OAJfDZ6z/bM8WuB/xnwWcf0mDf0Hj7fGCvRq4/PT7mzgbb7y+47r8XOa8XsCHef3WDfafG21e28M91PdCzyP7L4/1vhK5BvfTSK+xLI8giIo37D3ffVGT7b+L37USPWzT0B6LmeC/giFbc7/H4/eRG9t8JPEX0aMbDwPVABVEjP8TdvRX3+gAz25fosQqAHzdyvQlNXOL8+P0n7r6tkWNmxu//2sj+VQXHvMfdNwB3xz9+vYkMLXGPu28ssv2X8XsfM+uym/cQkXZMC4WIiDRuSSPb34zfV7p7TcOd7r7TzDYQPev6vudZ4yb0h8AZRKPA3YE9G1ziwGI3dXc3swviXBXxC2CYu69t/uM063jAiJ7l/X0jGd4wsyrgkMLtZtYBODH+8VYz+/EHTo7Uf9ZDGtn/uyYa/d8BVwHHmlmZu29v5LjmvNLI9r8V/HMP4O1dvL6ItHNqkEVEGreuke3vNrO/8Jj3ZpQws48SPbqwf8FxbxE9W+xAGVFD3ejopbuvi599rh9N/bm7P9pEjtbYL37f7O5NNYd/44MN7r5E+ev/uTmdm7h2U/eFqMn+ENGX8HbFW8U2uvs7Zlb/427PBCIi7ZcesRARSc8MouZ4IfBFYB937+bu+7v7AcC58XHW2AXMbE/gOwWb+gd4HKBYvsK/Tz7u7tbcK6H7iogkTg2yiEgK4pkpTiQaWf43d3+qyOMZ+3/wzA+4kugZ5c1AFXAk0ewQSaiO37ub2d5NHFdsruCN/HPU/KO7kaHo4yUN7vsu/5zaTUQkcWqQRUTScXD8Xu3ujT1GcFpTFzCzcuDa+MdLib4U58D3zezMBDK+Gl9vD6I5jItl6EM0p/H7eDQVXWX8Y1PzHDfnsy3Y91qD54931sfbjfuKiLxHDbKISDo2x+/7m9mHG+40s+OIFgspyszqZ67oCPyXuz/k7vOI5moGuNfMeu1OQHf/O9Ez0gBjrOCB3AJXNnGJ++P3r5nZwCaOo4nFOA4zs28XOX5f4KL4x5832L0lfu/e1D1FRFpKDbKISDqWAmuIRjlnmdkREC2uEa8s9z9EC4k05sfAMURfDPxBwfargD8RLcRxTwI5ryMaRf48cL+Z7R/n7G5mE4ia1C2NnHsv8BLR3y2/MrMRcWNLfI0Pm9m3zexZYEQj19gM/NTMBsUzY2Bm/Yimt9uPaAaR6Q3OWQ7sIHo05Gut/LwiIh+gBllEJAXuvhMYTvQ4wKnAcjPbQtQU/4JoOebLip1rZv8KXBL/+N3COXzj+YYHEc3JfI6ZDdnNnL8Hroh//A6wzsz+TvSM8ViieZ9fbeTcHUQr0v2BaNW724ANZvZ3M3uLaNaJmUSPSjQ2ldtdRNPYPQTUmNlm4I9EU9ptBc519/c9fxzPuPGz+Mf/MrNNZrYyfu3unMkikkNqkEVEUuLus4HPEY0Wv0X0uMQq4BaiOYjXNDwnfhRhBtHI83R3f7LIdRcRjfwCTDOzw3Yz52SieZrnETXwHYieL/6Ou1/ezLlvEjXA/w78mmjEt2ucfxnRKPOZNL7gyDaixUquJ/rdlBF9efD/AuXu/lwj5/0AuJlome+9iJbvPjS+t4hIq9huLrwkIiKy28zsfqIvHY539+vCphGRvNMIsoiIiIhIATXIIiIiIiIF1CCLiIiIiBToEDqAiIgkz8zWt/KUW9z9ljYJIyLSzqhBFhEpTS1ZtrpQ0Nke3H0IMCRkBhGReprFQkRERESkgJ5BFhEREREpoAZZRERERKSAGmQRERERkQJqkEVERERECqhBFhEREREp8P8BE6Rc6u+Q2kYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x504 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(nrows = 1, ncols = 1, figsize = (10,7));\n",
    "\n",
    "marker_style = dict(color='tab:blue', linestyle=':', marker='o',\n",
    "                    markersize=15, markerfacecoloralt='tab:red')\n",
    "\n",
    "ax.plot(max_depth_range,\n",
    "        accuracy,\n",
    "        lw=2,\n",
    "        color='k',\n",
    "        zorder = 0)\n",
    "\n",
    "s = ax.scatter(max_depth_range[2],\n",
    "           accuracy[2],\n",
    "           color = 'r',\n",
    "           s = 200,\n",
    "           alpha = 1,\n",
    "           zorder = 10,\n",
    "           marker = 'o',)\n",
    "\n",
    "s.set_edgecolor( 'black' )\n",
    "\n",
    "\n",
    "\n",
    "ax.set_xlim([1, 5])\n",
    "ax.set_ylim([.50, 1.00])\n",
    "ax.grid(True,\n",
    "        axis = 'both',\n",
    "        zorder = 1,\n",
    "        linestyle = ':',\n",
    "        color = 'k')\n",
    "\n",
    "yticks = ax.get_yticks()\n",
    "\n",
    "y_ticklist = []\n",
    "for tick in yticks:\n",
    "    y_ticklist.append(str(tick).ljust(4, '0')[0:4])\n",
    "ax.set_yticklabels(y_ticklist)\n",
    "ax.tick_params(labelsize = 18)\n",
    "ax.set_xticks([1,2,3,4,5])\n",
    "ax.set_xlabel('max_depth', fontsize = 24)\n",
    "ax.set_ylabel('Accuracy', fontsize = 24)\n",
    "fig.tight_layout()\n",
    "fig.savefig('../images/max_depth_vs_entropy.png', dpi = 300)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Depth is Not Always Equal to Max_Depth"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [],
   "source": [
    "from subprocess import call\n",
    "#call(['dot', '-Tpng', 'tree.dot', '-o', 'tree.png', '-Gdpi=600'])\n",
    "\n",
    "# List of values to try for max_depth:\n",
    "max_depth_range = list(range(1, 6))\n",
    "\n",
    "# List to store the average RMSE for each value of max_depth:\n",
    "\n",
    "accuracy = []\n",
    "depth_list = []\n",
    "for max_depth in max_depth_range:\n",
    "    \n",
    "    clf = DecisionTreeClassifier(max_depth = max_depth, \n",
    "                             random_state = 0)\n",
    "    clf.fit(X_train, Y_train)\n",
    "    score = clf.score(X_test, Y_test)\n",
    "    depth = clf.get_depth()\n",
    "    depth_list.append(depth)\n",
    "    accuracy.append(score)\n",
    "    \n",
    "    outputFileDot = \"../images/BadDepthExample\" + str(max_depth)+'_actual'+str(depth)+ \".dot\"\n",
    "    outputFilePng = \"../images/BadDepthExample\" + str(max_depth)+'_actual'+str(depth)+ \".png\"\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Feature Importance\n",
    "\n",
    "Scikit-learn allows you to calculate feature importance which is the total amount that Gini index or Entropy decrease due to splits over a given feature"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [],
   "source": [
    "clf = DecisionTreeClassifier(max_depth = 3, \n",
    "                         random_state = 0)\n",
    "clf.fit(X_train, Y_train)\n",
    "\n",
    "score = clf.score(X_test, Y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>feature</th>\n",
       "      <th>importance</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>petal width (cm)</td>\n",
       "      <td>0.578</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>petal length (cm)</td>\n",
       "      <td>0.422</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>sepal length (cm)</td>\n",
       "      <td>0.000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>sepal width (cm)</td>\n",
       "      <td>0.000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             feature  importance\n",
       "3   petal width (cm)       0.578\n",
       "2  petal length (cm)       0.422\n",
       "0  sepal length (cm)       0.000\n",
       "1   sepal width (cm)       0.000"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "importances = pd.DataFrame({'feature':X_train.columns,'importance':np.round(clf.feature_importances_,3)})\n",
    "importances = importances.sort_values('importance',ascending=False)\n",
    "importances"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If a feature has a low feature importance value, it doesnt necessarily mean that the feature isnt important for prediction, it just means that the particular feature wasnt chosen at a particularly early level of the tree. Could be that the feature could be identical or highly correlated with another informative feature. Feature importance values dont tell you which class they are very predictive for or relationships between features which may influence prediction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Decision Path\n",
    "Can also explore other features of decision trees"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.tree.export import export_text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "|--- petal width (cm) <= 0.80\n",
      "|   |--- class: 0\n",
      "|--- petal width (cm) >  0.80\n",
      "|   |--- petal length (cm) <= 4.95\n",
      "|   |   |--- petal width (cm) <= 1.65\n",
      "|   |   |   |--- class: 1\n",
      "|   |   |--- petal width (cm) >  1.65\n",
      "|   |   |   |--- class: 2\n",
      "|   |--- petal length (cm) >  4.95\n",
      "|   |   |--- petal length (cm) <= 5.05\n",
      "|   |   |   |--- class: 2\n",
      "|   |   |--- petal length (cm) >  5.05\n",
      "|   |   |   |--- class: 2\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(export_text(clf, feature_names=data.feature_names))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualizing Decision Tree \n",
    "Image Here\n",
    "If you are curious about how to visualize your decision tree, please see my tutorial. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is something I will cover in a future tutorial"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [conda env:decisionTree]",
   "language": "python",
   "name": "conda-env-decisionTree-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
